llms.txt is a plain-text Markdown file you place at the root of your domain (yoursite.com/llms.txt) that gives AI systems a clean, curated map of your most important content. Think of it as robots.txt, but instead of telling crawlers what to avoid, it tells large language models what matters and where to find it.
What is llms.txt, exactly?
The idea was proposed in 2024 by Jeremy Howard, co-founder of Answer.AI and fast.ai, and the specification lives publicly at llmstxt.org. It exists because a normal web page is a messy thing for a model to read — wrapped in navigation, scripts, cookie banners and markup that have nothing to do with your actual content.
An llms.txt file strips all that away. It is written in standard Markdown: a single H1 with your site or business name, an optional one-line summary in a blockquote, and then H2 sections containing curated lists of links with short descriptions. A model (or an AI search engine) can read it in one pass and immediately understand what you do and which pages are worth citing.
It is not a magic ranking lever, and it does not replace good on-page content. It is a signpost — a deliberate, human-curated answer to the question "if a model only read one file about this business, what should it say?"
That framing matters. Most of your site is written for a person clicking through pages. The llms.txt file is written for a machine that gets one read and then has to summarise you to someone else. Deciding what goes in it is a useful exercise even before the file does anything: it forces you to name the three or four pages that actually represent your business.
Who actually uses it right now?
Here's the honest part: llms.txt is a proposed standard, not an official one, and adoption is still early. As of 2026 there is no public confirmation that Google, OpenAI or Anthropic use the file to train or ground their models. Several developer-tooling companies — including Anthropic, Stripe and others in their documentation — publish one, largely so their docs are easy for coding assistants to ingest.
So why bother now? Because the cost is near zero and the downside is none. The file is honest, machine-readable signage about your own site, and the sites publishing it tend to be exactly the technically-sharp ones that show up in AI answers. Pulling a lever almost nobody has pulled is cheap insurance — and if the standard does get adopted, you're already compliant instead of scrambling later.
robots.txt |
llms.txt |
|
|---|---|---|
| Audience | Search crawlers | LLMs / AI engines |
| Purpose | What to avoid | What matters |
| Format | Plain directives | Markdown |
| Status | Long-standing standard | Proposed (2024) |
How do I write an llms.txt file?
An llms.txt file is a single Markdown document at your domain root that points AI systems to your best content in a fixed, predictable shape. Start with one # H1 holding your business name, add an optional one-sentence summary as a > blockquote, then create ## sections — most commonly "Key pages," "Services" and "Contact" — each holding a bulleted list of links written as [label](url): one-line description. Keep every description factual and current; the file is only useful if it's true. The concrete first step: open a text editor, write the four sections below, save the file as llms.txt, and upload it to your web root so it resolves at yoursite.com/llms.txt. Verify by visiting that URL in a browser — you should see raw text, not a styled page. The whole job takes about twenty minutes, and unlike most SEO work there's nothing to wait on and no third party to approve it.
Here's a copy-paste starting point for a service or industrial site:
# IndexLifts — done-for-you SEO & AI search for service & industrial firms
> We help service and manufacturing businesses get found in Google and
> named in AI answers (ChatGPT, Perplexity, Google AI Overviews).
## Key pages
- [Services](https://indexlifts.com/services.html): what we do and how engagements work
- [Industries](https://indexlifts.com/industries.html): the sectors we serve
- [About](https://indexlifts.com/about.html): the team and how we measure results
## Contact
- Email: hello@indexlifts.com
A few rules that keep it useful:
- One file, at the root, served as plain text.
- Link only to pages you'd be happy to be quoted from.
- Use real, absolute URLs and honest one-line descriptions.
- Update it whenever your key pages or offering change.
What are the common mistakes to avoid?
Because the format is so simple, the failure modes are simple too — and they're easy to dodge once you know them.
- Treating it as a sitemap. A
sitemap.xmllists every URL for crawlers;llms.txtis a hand-picked shortlist of your best pages with context. Dumping 400 links in defeats the purpose. Pick the ten that matter. - Letting it go stale. A file that points to a discontinued service or a 404 is worse than none — it actively misinforms anything that reads it. Review it whenever your site changes.
- Writing marketing fluff. "World-class, best-in-class solutions" tells a model nothing. Write the plain, factual description a careful human would.
- Serving it wrong. It must return as
text/plain(ortext/markdown) at the exact root path. If your CMS wraps it in a styled HTML template, the value is lost. - Expecting instant rankings. It's a signpost, not a ranking factor. Pair it with genuinely good, factual on-page content.
The optional companion file, llms-full.txt, goes further by inlining the full text of your key pages into one document. It's useful for documentation-heavy sites, but for most service businesses the short llms.txt is plenty.
Why does this help AI read my site?
Most sites bury their best facts inside navigation, scripts and PDFs, which forces an AI engine to render and guess. An llms.txt file hands the model a short, clean summary and a list of the pages you most want cited — no rendering, no parsing, no guessing. It won't override weak content, but for a site whose facts are sound it removes the friction between "we said it" and "the model found it."
The takeaway
llms.txt is low-effort, honest signage for the AI era: a twenty-minute Markdown file that tells models who you are and which pages to trust. It is still an early, proposed standard, so treat it as cheap insurance rather than a guaranteed ranking win — publish one, keep it accurate, and update it when your business does.
