Does Anything Actually Read llms.txt? Three Measurements Say No

The short answer is that almost nothing reads it. Three separate measurements published this year point the same direction, and the largest of them found that 97% of the llms.txt files it tracked received zero fetches in a single month. If you are deciding today whether to spend real money on this, that number is the one to start from.

For anyone who has not met the idea: llms.txt is a proposed convention where you publish a Markdown file at the root of your domain listing the pages you most want a language model to read. It is not a standard. No engine has committed to honouring it.

Three measurements, all pointing the same way

Ahrefs looked at 137,000 domains in May 2026. Twenty-eight percent of them published an llms.txt. Of those files, 97% were fetched zero times that month. Among the fetches that did happen, 96% came from bots of some kind, and 19.5% came from named AI tools. So the file is being published far more often than it is being read, and the reading that happens is a rounding error against the publishing.

SE Ranking ran a study across 300,000 domains and concluded the file "doesn't impact how AI systems see or cite your content today." The study then recommends adding it anyway, calling it "a low-effort way to prepare for the next wave." A commenter on the thread discussing it put the obvious question well: "I don't follow? 'It does nothing, prepare for tomorrow by using it'?" That tension runs through most of the pro-llms.txt writing, and it is worth noticing when you read it, because the recommendation is usually doing work the evidence is not.

An audit of Adobe Experience Manager sites went at it from the other end: 30 days of raw CDN logs across 1,000 domains. GPTBot, ClaudeBot and PerplexityBot did not appear at all. Google's desktop crawler accounted for 95% of hits. Bing made seven requests in the whole window, all of them concentrated on a single domain out of the thousand. OpenAI's search bot made ten calls.

Weigh each for what it is. The Adobe figure is one CDN, one month, and enterprise AEM sites are not a random sample of the web. The Ahrefs number is a crawl-side view rather than a server-side one. But they were measured independently, by parties with no shared method, and they agree. That is about as much as this kind of question gets.

Google says two different things because it is answering two questions

This is where most of the argument comes from, and it dissolves once you see it.

Search Central's guidance on AI optimization says you do not need special files for AI. Chrome Developers' Lighthouse documentation on agentic browsing says an llms.txt helps agents understand a site. Both are Google. Both are current. People screenshot one at the other and the thread goes nowhere.

They are not the same question. Search Central is talking about generative search visibility: whether your pages get surfaced and cited in an AI answer. For that, the evidence above holds, and Google's answer is no, you do not need the file.

Lighthouse is talking about agentic browsing: a piece of software acting on a user's behalf, arriving on your site to do something, needing a map. That is the WebMCP direction, and it is a different problem with different failure modes. A hint file makes more obvious sense there, because an agent trying to complete a task has a reason to look for one.

If you conflate the two, Google looks incoherent. If you keep them apart, the guidance is consistent and only one of the two questions is the one most people are actually asking. An SEO who attended a live session at Google's Toronto office reported that the question was asked from the floor, and that the answer was no, llms.txt is neither needed nor required, and having one neither helps nor hurts. That is one attendee's account rather than a published statement, so weigh it accordingly. It matches what Search Central puts in writing.

The best argument for it, taken seriously

Robots.txt was proposed around 1994 and did not become an official internet standard until RFC 9309 in 2022. As one commenter put it, "it took robots.txt nearly 28 years to be adopted as an internet standard. llms.txt was proposed a little under 2 years ago."

That is a real point and it should temper any confident dismissal. Conventions get adopted slowly, and the ones that win rarely look inevitable at year two. Nothing in the fetch data proves llms.txt will not be read in 2029.

What the data does prove is that it is not being read now, which is the question you are answering when you decide this quarter's budget.

The thing nobody in the argument talks about

The sharpest thing anyone said in these arguments came from a sceptic: "all of the propaganda agents talk about the file, not the contents... Nobody talks about what the content should be, and that's the red flag."

Go and read a dozen llms.txt articles and check. Almost all of them tell you where to put the file and what the syntax looks like. Almost none argue about what belongs in it, which pages earn a place, how you would know if you chose wrong. That is what advice looks like when the recommendation is about the gesture rather than the outcome.

Meanwhile, the thing people hope llms.txt does is already done by a file that engines demonstrably honour. AI crawler access is controlled in robots.txt, with named user-agents that show up in your logs when they arrive. If you want to see what a site actually tells crawlers, our robots.txt fetcher pulls the file and shows it to you. Start there if the underlying goal is control over what reads you.

So do you write one

Yes, if it costs you five minutes and you never think about it again. One site owner in these threads summed up the honest version: "it took me 5 minutes to create it... I don't expect any results."

No, if it becomes a line item, a monthly deliverable, or a thing you maintain. There is no measurable return today to justify the maintenance, and a file that drifts out of date is worse than no file.

This site has argued before that GEO effect cannot be traced end to end, which puts the spending in the brand budget and means it gets judged the way brand spend is judged: on belief about the future, at a size you can afford to be wrong about. llms.txt is the cheapest possible instance of that bet. Five minutes of belief is defensible. A retainer is not.

One caution against the "no downside" line, because it is the argument you will hear most. A commenter answered it well: "there's always a downside to parroting myths, it makes people do and reward the wrong thing." The file is free. Recommending it to a client as though it works is not.

What would change this verdict

Your own server logs are the first place a change would show. If GPTBot, ClaudeBot or PerplexityBot start requesting /llms.txt on your domain with any regularity, the picture has moved and you will see it before any study reports it. That is a log filter, not a project.

A second signal would be any engine publishing documentation that it reads the file and says how it uses it, rather than a third party inferring it. Right now no engine has.

The third would be a replication of the Ahrefs and Adobe measurements showing fetch rates in a different order of magnitude. One month of near-zero is a snapshot. A year of it is a verdict, and a reversal would be equally visible.

Until one of those happens, treat llms.txt as five minutes of insurance against a future that has not arrived.