Available
llms.txt: what is this file for language models?
A small text file at the root of a site, written for AI tools rather than for people.
llms.txt is a Markdown text file placed at the root of a website (/llms.txt) that briefly introduces the business and lists its most useful pages, so large language models can find their way quickly. It is an open proposal from 2024 rather than an official standard, and it sits alongside robots.txt and the sitemap.
In short
- Proposed by Jeremy Howard on September 3, 2024.
- The only required element is an H1 heading with the site or project name.
- The proposal also suggests a .md version of each useful page.
- It complements robots.txt and the sitemap; it replaces neither.
Who proposed llms.txt, and why?
Jeremy Howard published the proposal on llmstxt.org on September 3, 2024 [1], and the project lives in a public repository [2]. The reasoning is simple: a large language model reading a website has to wade through menus, scripts and ads. A short Markdown file hands it the essentials in one pass.
The official page describes itself as a proposal open to community input [1]. No standards body has adopted it, and no search engine is obliged to read it.
How is an llms.txt file structured?
The proposal sets a specific order [1]. An H1 with the project or site name is the only mandatory part. Then comes a blockquote summary, optional paragraphs, and H2 sections holding lists of links written as [name](url), optionally followed by a colon and a note. By convention, a section called “Optional” holds secondary links that a tool can drop when it needs less context.
The file goes at the root (/llms.txt) or in a subfolder such as /docs/llms.txt, where it covers that folder's pages [1].
| File | Job | Status |
|---|---|---|
| robots.txt | Tell crawlers which URLs they may crawl | IETF standard (RFC 9309) |
| XML sitemap | List pages and when they were last updated | sitemaps.org protocol |
| llms.txt | Summarize the site and point to useful pages, in Markdown | Open proposal (2024) |
According to llmstxt.org, llms.txt is designed to coexist with robots.txt and sitemap.xml.
What does an llms.txt look like for a service business?
What are the .md page versions?
The proposal also asks sites to serve a Markdown copy of each useful page at the same address, either by appending .md (page.html.md) or by replacing the extension (page.md). A page can advertise that copy with a rel="alternate" link of type text/markdown [1]. Same idea: give tools clean text without the page chrome.
Which llms.txt mistakes should you avoid?
For context, robots.txt is defined by IETF RFC 9309 [4] and the XML sitemap by the sitemaps.org protocol [5]; llms.txt has no comparable standing.
- Treating it as a robots.txt replacement; it neither blocks nor allows any crawler.
- Listing pages that your XML sitemap or robots.txt exclude, so the signals contradict each other.
- Writing an ad (“the best in Quebec”) instead of a factual summary.
- Forgetting to update it after a redesign, leaving dead links behind.
- Assuming Google needs it; Google says there are no additional requirements for its AI features [3].
What does Pilotiko do with llms.txt?
For the site it builds and hosts, Pilotiko generates the llms.txt file automatically, along with the sitemap, robots.txt and structured data. You don't have to write it by hand. It is one piece of GEO work, not a promise of being cited.
Frequently asked questions
Is llms.txt mandatory?
No. It is an open proposal with no standard status. It costs almost nothing to publish, but leaving it out doesn't stop a site from being indexed.
Does Google read llms.txt?
Google's documentation on its AI features doesn't mention it and says no special optimization is needed. Don't rely on llms.txt to get into AI Overviews.
How is llms.txt different from robots.txt?
robots.txt tells crawlers what they may crawl, so it is an access rule. llms.txt summarizes content and suggests links, so it is a reading aid with no effect on access.
Where should the llms.txt file go?
At the root of the domain, at /llms.txt. The proposal also allows one in a subfolder to cover just that folder's pages.
What belongs in the Optional section?
Useful but secondary links such as careers, policies or archives. An AI tool can skip them when it needs a shorter context.
Sources
- llmstxt.org, The /llms.txt file (accessed on October 15, 2026).
- Answer.AI (GitHub), llms-txt (dépôt du projet) (accessed on October 15, 2026).
- Google Search Central, AI features and your website (accessed on October 15, 2026).
- IETF (RFC Editor), RFC 9309 : Robots Exclusion Protocol (accessed on October 15, 2026).
- sitemaps.org, Sitemaps XML format (accessed on October 15, 2026).
Read next
robots.txt: what does this file tell search engine crawlers?
robots.txt tells crawlers which URLs they may visit. How RFC 9309 rules are read, what happens on errors, a small business example and the costly mistakes.
XML sitemap: what does it do for search engines?
What an XML sitemap is, which tags it uses, the official 50,000-URL limit, whether a small business site needs one and the mistakes that confuse search engines.
GEO: what is generative engine optimization?
Generative Engine Optimization explained: where the term comes from, what research and Google say about AI answers, and a scored page check for a Quebec SME.
Large language model (LLM): how does it work?
What a large language model is, how it writes by predicting tokens, where it helps a service SME, a worked example and the limits to know before adopting one.