What is PerplexityBot?
PerplexityBot is Perplexity’s crawler that surfaces websites and links them in Perplexity search results. According to Perplexity, it is not used to crawl content for training AI foundation models.
Perplexity is an answer engine that backs almost every answer with sources. For a page to qualify as a source, it has to be discoverable, and that is PerplexityBot’s job. Perplexity explicitly recommends allowing it in robots.txt and permitting requests from its published IP ranges.
The second half of that recommendation is often missed. A firewall can block requests even when robots.txt allows them. Perplexity’s documentation notes that site owners using a web application firewall may need to explicitly allow its bots.
Besides PerplexityBot there is Perplexity-User, which fetches pages when a person asks a question. The two follow different rules. Blocking PerplexityBot affects whether your site shows up in Perplexity search, not every single fetch triggered by a user request.
What it means for your website
Check that your robots.txt allows PerplexityBot and that your host or CDN lets requests with this user agent through. The deeploupe GPTBot check reads the robots.txt rule for PerplexityBot. It does not fetch your page with the PerplexityBot user agent, so a firewall block against exactly this bot would not show up there. Your server logs answer that question.
More on deeploupe
Sources
Terms help you understand. Whether AI crawlers can reach your site is something you measure.
Check your website for freefree · no signup · no credit card