What is OAI-SearchBot?
OAI-SearchBot is OpenAI’s crawler that indexes websites for the search features in ChatGPT. According to OpenAI, sites that disallow OAI-SearchBot in robots.txt will not be shown in ChatGPT search answers; the crawler is not used to train models.
While GPTBot collects content for model training, OAI-SearchBot makes pages eligible to appear in ChatGPT search results and to be linked as a source. The two crawlers can be controlled independently in robots.txt, so a site can opt out of training and still remain reachable for search.
For the question whether a page can appear as a source in current ChatGPT answers with web search, OAI-SearchBot is therefore the more decisive crawler. OpenAI’s documentation also notes that even with a block, a plain navigational link to the site may still appear in answers, for example when someone asks about the website itself.
As with every crawler: a permissive robots.txt does not help if a firewall or bot protection rejects the requests at the server level.
What it means for your website
Check your robots.txt for whether OAI-SearchBot is excluded explicitly or through a general rule for all crawlers, and look in your server logs for whether requests with this user agent are answered successfully. deeploupe’s free GPTBot check does not test OAI-SearchBot; it tests GPTBot and ClaudeBot live. For OAI-SearchBot, robots.txt and logs remain the way to check.
Related terms
More on deeploupe
Sources
Terms help you understand. Whether AI crawlers can reach your site is something you measure.
Check your website for freefree · no signup · no credit card