Last updated: September 19, 2026
Does LLMs.txt help? For Google Search, including AI Overviews and AI Mode, the answer is a documented no: Google’s own guidance says the file neither helps nor harms visibility because Search ignores it. For the wider set of AI systems, the answer is narrower than the advice industry suggests: the largest access studies find that almost no crawler fetches the file, and the one large study that tested for an effect on AI citations found none. Where the file earns its keep is a specific, smaller job, giving AI coding tools and agents a compact map of documentation. This guide sets out what the file is, what the evidence shows, how it differs from the other machine-readable layers people confuse it with, how to generate one, and a short decision rule for whether your site should bother.
The clearest statement is Google’s. As Search Engine Journal reported, Google’s Search team has said for more than a year that llms.txt is not something it uses or plans to adopt; John Mueller compared it to the keywords meta tag and noted that bots did not request the file, Gary Illyes confirmed at Search Central Live that Google was not pursuing it, and Google’s optimisation guide for generative AI features lists it among the things to skip. On 15 June 2026 Google added a section to that guide, headed “Clarifying guidance on llms.txt files”, stating that the file is not needed to appear in Google Search and that keeping one will neither harm nor help a site’s visibility or rankings because Google Search ignores it.
If you would rather put the effort into the layers Google and the AI engines do read, sign up for free and NytroSEO will audit and align the titles, descriptions and structured data on every page in your sitemap.
The access data points the same way. According to a summary of the major studies, Ahrefs tracked 137,210 domains with a valid llms.txt and found that 97 per cent received zero requests for the file in May 2026; SE Ranking’s analysis of roughly 300,000 domains found adoption of about 10 per cent and no statistical relationship between having the file and being cited by AI systems, with only one of the fifty most-cited domains carrying it at all. Both are third-party studies with disclosed methods, and both measure different things, access and correlation, that agree.
Lee Agam, founder and CEO of NytroSEO, sees llms.txt as a case study in how a plausible idea becomes a checklist item before anyone measures it. In his view the file’s appeal is that it is cheap and visible, which is exactly why it spread through agency deliverables and audit tools, and the evidence now shows that for most sites the bots never arrive. His advice is to treat it like any other self-declared signal: if a crawler has to read your pages anyway to verify what the file claims, the file is a shortcut for the crawler at best and an unmaintained liability at worst. Sites with real developer documentation should ship it, keep it current and stop expecting it to move search or citations; every other site should spend the same hour on indexability, metadata and structured data, which the engines demonstrably use.
What is an LLMs.txt file?
An LLMs.txt file is a plain-text markdown document placed at the root of a domain, at yoursite.com/llms.txt, that summarises what the site is and links to its most important pages so that a language model or agent can find key content without crawling everything. The convention was proposed in September 2024 by Jeremy Howard and is documented at llmstxt.org; a companion llms-full.txt can carry the full text of the linked pages in one file.
The structure is deliberately simple: an H1 with the site name, a short blockquote summary, optional paragraphs of context, and H2 sections containing lists of links with one-line descriptions. Nothing in it is enforced by any standards body, no major search engine has committed to reading it, and there is no equivalent to the robots.txt convention that crawlers are expected to honour. It is a signpost, not a control.
It is also frequently confused with things it is not. It is not a way to allow or block AI crawlers; that is robots.txt and, where offered, CDN bot controls, which our guide to allowing AI crawlers while blocking AI training covers. It is not structured data; schema.org markup describes entities on a page and is read by search engines. And it is not a substitute for indexable HTML, which every AI system that cites the web depends on.
Does LLMs.txt help? The evidence in 2026
Three kinds of evidence answer the question, and they converge.
The chart above summarises the access, adoption and correlation findings; the sources are named on the image and below.
Google’s position is explicit and has been consistent since 2024. Search does not use the file. AI Overviews and AI Mode do not use the file. Google’s June 2026 wording is softer in tone, “completely fine” to create one, and unchanged in substance: it will neither harm nor help. When an llms.txt briefly appeared on Google’s own developer documentation in December 2025, Mueller confirmed it was not an endorsement; an internal content system had added the file automatically, and the Search Central copy was removed within hours.
Access data shows the bots mostly do not come. Ahrefs’ 137,210-domain study found 97 per cent of files received no requests in a month; of the small share that were fetched, about a fifth of requests came from named AI tools, with GPTBot first and Claude Code second. A separate log analysis cited in the same coverage, spanning hundreds of millions of AI bot events over ninety days, found only a few hundred requests for the file; the major AI crawlers overwhelmingly fetch HTML pages instead.
Correlation data shows no effect. SE Ranking’s model found that having the file did not predict AI citations, and adoption was almost identical across low, mid and high-traffic sites, so it is not a practice that high performers share and others copy.
One more data point closes the loop on the ranking question. The domains that are cited most by AI systems are, overwhelmingly, domains without the file: in SE Ranking’s data only one of the top fifty carried it. If the file were a meaningful input to citation, the most-cited sites would be the first to have discovered it. They have not, because their citations come from the same place everyone else’s do, pages that answer clearly and can be fetched.
What the evidence does not say is that the file is useless everywhere. The requests that do arrive come from coding tools and agents reading developer documentation, and one visible consumer exists: Chrome’s Lighthouse includes an agentic-browsing audit that checks for the file. Mueller’s own summary is the fairest one, a temporary crutch, perhaps to save some tokens, for AI coding tools reading docs.
LLMs.txt vs OKF vs schema: what each layer does
Because three machine-readable layers get bundled together in advice, it is worth separating them.
LLMS.TXT is a public signpost for AI tools: a markdown index at the root. Read by some coding agents; ignored by Google Search; no documented use by the major answer engines’ citation systems.
Schema.org structured data is what search engines actually read. Organization and Person markup tell Google who is speaking, Article and Product markup describe the content, and Google’s AI features draw on the same index and understanding. Our guide to getting AI-search ready with metadata and structured data covers what to prioritise.
The Open Knowledge Format, published by Google Cloud in 2026, is a bundle of linked markdown files with YAML metadata that an organisation’s own AI agents read internally. It is a knowledge-management format, not a search signal, and it is not something a public website needs unless it runs agents over its own documents.
The practical hierarchy is the reverse of the attention each gets. Indexable, renderable HTML first, because every engine and assistant reads it; our guide to JavaScript SEO for AI agents explains why rendering matters more than any file. Accurate metadata and structured data second. LLMs.txt a distant third, for sites with documentation an agent might consult. OKF only if you build internal agents.
How to generate an LLMs.txt file? And how to keep it honest
If the decision rule in the next section says yes, the file takes an hour and should look like this.
Write it by hand or from a template. Start with the site name as an H1, a one-sentence summary in a blockquote, and a short paragraph of context. Then add H2 sections, Documentation, Product, Pricing, Policies, each containing a list of links with a one-line description per link. Keep it to the pages an agent would need to answer a question about the product, typically ten to fifty links, not the whole sitemap.
Generators and plugins exist for most platforms, and several documentation tools emit the file automatically. Whichever route you use, the file must be plain text, served with a text content type, at exactly website.com/llms.txt, and it must return 200.
Two additions are worth more than the file itself for agent readability. A plain markdown copy of the pricing page, with tiers, limits and what is included, so that an agent asked “what does this cost” gets a clean answer. And a visible last-updated line at the top of the file, because an agent has no other way to judge whether the map is current.
Test it the way a tool would. Fetch the file with a plain HTTP client and confirm the content type, the status code and that every link resolves. Then paste the file into a coding assistant and ask it to describe the product and its pricing from the file alone; if the answer is wrong or vague, the map is not doing its job.
Then maintain it or remove it. A file that lists pages that have moved, prices that have changed or products that no longer exist is worse than no file, because the few tools that do read it will treat it as the truth. Our earlier note on how llms.txt fits ChatGPT search optimisation described the format when it was new; the recommendation here supersedes it on expected impact.
Does LLMs.txt help? When to bother, when to skip
The flow above ends in three outcomes; most commercial sites reach the third.
Bother if you publish developer documentation, an API reference or product docs that engineers consult through coding tools, and you have someone who will keep the file current. Stripe, Vercel, Cloudflare and Anthropic ship one, and they are exactly that kind of site. Add the pricing markdown alongside it.
Consider it if you sell software with a documented setup process and your support team fields questions that an agent could answer from a short map. Expect no effect on Google or on citations; expect a small convenience for the tools that read it.
Skip it, for now, if you are a publisher, an ecommerce store, a local business or an agency site without developer documentation. Nothing in the evidence suggests the file will affect your rankings, your AI impressions or your citations, and the hour is better spent on the layers that do: crawler access, indexability, one clear answer per page, accurate titles and descriptions, consistent Organization and Person markup. If a tool or a consultant lists llms.txt as an AI visibility tactic for a site like that, ask them for the access logs.
Whatever the decision, do not let the file distract from the measurement that matters. The Search Console Generative AI report now shows which pages earn impressions in AI Overviews and AI Mode; a prompt panel across ChatGPT, Perplexity, Gemini and Copilot shows citations. Neither will move because of a markdown file at the root. They move when the pages themselves are reachable, readable and clearly answer the question, which is the work NytroSEO automates across a site. Book a strategy meeting with the NytroSEO team if you want the machine-readable layers of a large site or a client portfolio prioritised in the order the evidence supports.
Does LLMs.txt help? A decision rule in one line
Ship llms.txt if an engineer’s coding tool would benefit from a map of your docs and someone will maintain it; otherwise spend the hour on indexable HTML, metadata and schema.
Frequently Asked Questions
No. Google’s AI optimisation guide, updated on 15 June 2026, states that llms.txt files are not needed to appear in Google Search, including AI Overviews and AI Mode, and that keeping one will neither harm nor help visibility or rankings because Google Search ignores them. Google’s Search team has said the same since 2024.
An LLMs.txt file is a plain-markdown document at the root of your domain, such as yoursite.com/llms.txt, that summarises what your site is about and links to its most important pages so AI tools can find key content without crawling everything. It is a signpost for agents, not a crawler control and not structured data.
No. LLMs.txt is a public signpost for AI tools. The Open Knowledge Format, published by Google Cloud in 2026, is a bundle of linked markdown files with YAML metadata that an organisation’s own AI agents read internally. Neither is used by Google Search as a ranking or visibility signal.
Write it by hand in markdown with the site name, a one-sentence summary, a short context paragraph and grouped links to ten to fifty key pages, or use a generator or documentation tool that emits it. Serve it as plain text at website.com/llms.txt, add a visible last-updated line, and remove it if it goes stale.
No. Sites with developer documentation that coding tools consult can benefit modestly, and the file is harmless if maintained. Publishers, stores, local businesses and most commercial sites should prioritise indexable HTML, accurate metadata and structured data, which search engines and AI features actually use, because no evidence links the file to rankings or citations.
Does LLMs.txt help? Not enough to replace the layers the engines read
The evidence on LLMs.txt is settled enough to stop debating it. Sign up for free to have NytroSEO keep your titles, descriptions and structured data aligned across every page automatically, or book a strategy meeting if you manage a large site or a client portfolio and want the machine-readable layers audited in the right order.
