What is llms.txt?
llms.txt is a proposed file format in which a website places a Markdown overview of its most important content at /llms.txt for language models. It is not a standard, and there is no evidence that AI crawlers use the file.
The proposal comes from Jeremy Howard and is described at llmstxt.org. The idea is that language models find a compact, readable summary of a site instead of working through HTML full of navigation and ads. Many SEO tools now recommend the file as an AI visibility measure.
The evidence does not support that recommendation. An analysis of crawler logs found llms.txt fetches by GPTBot, ClaudeBot and PerplexityBot only at statistically negligible levels. Google’s documentation on AI features says you do not need to create new machine readable files or AI text files to appear there. OpenAI controls crawling by its bots through robots.txt.
The file does no harm and takes little effort. As a lever for mentions in AI answers, however, it is currently unproven. deeploupe therefore rates it as a low impact measure in its own report.
What it means for your website
Add an llms.txt if it costs you little time, but do not expect a measurable effect. What matters more is that AI crawlers are allowed to fetch your site at all and that your core statements are in the delivered HTML. The free GPTBot check covers part of that. A detailed assessment with sources is on the German myths page about llms.txt.
Related terms
More on deeploupe
Sources
Terms help you understand. Whether AI crawlers can reach your site is something you measure.
Check your website for freefree · no signup · no credit card