What is ai.txt?
ai.txt is a proposed text file in a website’s root directory meant to state whether and how AI systems may use its content. It is not a recognised standard, the major AI crawlers do not document it as binding, and there is no evidence it affects citations.
The idea behind ai.txt resembles llms.txt: an extra file next to robots.txt addressed specifically to AI providers. Several independent proposals use the name, some as a reservation against training use, some as a pointer to content. None has been adopted by a standards body.
What matters is what the crawlers themselves document. OpenAI, Anthropic, Perplexity and Google describe controlling their bots through robots.txt and their user agent names. ai.txt does not appear in that documentation as a control mechanism. Google states explicitly that no special files or markup are needed for the AI features in Search.
That does not make the file harmful, and it costs little. It does not replace a robots.txt rule, though, and relying on it to become visible in AI answers means relying on a mechanism without evidence.
What it means for your website
To allow or block AI crawlers, write robots.txt rules with the documented names such as GPTBot or ClaudeBot, then verify that hosting or bot protection is not blocking access anyway. An ai.txt can sit alongside, but an audit should not flag its absence as a missing requirement.
Related terms
More on deeploupe
Sources
Terms help you understand. Whether AI crawlers can reach your site is something you measure.
Check your website for freefree · no signup · no credit card