Implementation · Contested

llms.txt, honestly

Most write-ups of this file are selling something. Here is what the measurement says, followed by what we still recommend and why those two things aren't in conflict.

01What the 2026 data shows

• Adoption is around one site in ten. An SE Ranking study of 300,000 domains found a 10.13% adoption rate — and among the fifty most AI-cited domains, only one had the file at all.
• Crawlers barely fetch it. A Limy.ai monitoring analysis of over 500 million AI bot events across a 90-day window found only a few hundred requests targeting /llms.txt directly. GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended overwhelmingly crawl HTML instead.
• Google has said no. Gary Illyes confirmed Google doesn't support llms.txt and isn't planning to; John Mueller compared it to the discredited keywords meta tag. Google's 2026 generative-AI documentation lists it among unnecessary tactics.
• No provider has committed. As of 2026, no major AI company has publicly committed to reading or acting on llms.txt in production.
• A large share of existing files are junk. The HTTP Archive's 2025 Web Almanac found around 40% of published files were plugin-generated defaults rather than deliberate, curated documents.

02So why ship one?

Three defensible reasons, none of which is “it will get me cited.”

1. The cost is near zero and the option is real. Twenty minutes buys you a position if agentic routing does standardise. That's a sensible asymmetric bet.
2. Writing it is a useful forcing function. Producing a one-sentence description of your forty most important pages surfaces duplication, orphaned content and pages that don't actually say anything. Several teams get more value from the audit than the file.
3. It's a business-to-agent surface, not an SEO artifact. The interesting use case isn't search citations — it's an agent trying to work out what your company offers and where the authoritative page for each thing lives.

03How to write one properly

The proposal specifies Markdown: an H1 with the site or brand name, a blockquote summary, optional free prose, then H2 sections listing links with a one-sentence description each. Keep it curated — ten to forty genuinely important pages, not a sitemap dump.

# Example Co

> One-paragraph description of what the organisation does,
> who it serves, and what makes its content authoritative.

Optional prose: scope of the site, what is and isn't covered,
licensing or citation preferences.

## Core guides

- [What is X](https://example.com/what-is-x): One sentence on what this page answers.
- [How X works](https://example.com/how-x-works): One sentence on what this page answers.

## Reference

- [Glossary](https://example.com/glossary): Definitions of 40 terms used across the site.

## Optional

- [Full text](https://example.com/llms-full.txt): Plain-text copy of every page.

Our own llms.txt and llms-full.txt are live and follow this shape. Copy them.

Rules of thumb

• Serve it as text/plain at the site root, exactly at /llms.txt.
• Use absolute URLs.
• Curate ruthlessly — the file's only advantage over a sitemap is editorial judgement.
• Include a last-reviewed date and actually keep it current. A stale file is worse than none.
• Don't duplicate every page as a Markdown mirror unless you handle indexation properly.

04How to check whether anything reads it

Don't guess. Filter your access logs for requests to /llms.txt and /llms-full.txt by known AI user agents. You can also embed a unique URL inside the file that appears nowhere else — a honeypot only an automated reader would follow — and watch for hits. Cloudflare's bot analytics will break this down by user agent without touching raw logs.