What llms.txt is
llms.txt is a markdown file a site can publish so an agent has a short map of the content. Jeremy Howard published the proposal on September 3, 2024. Version 2 of the text at llmstxt.org is dated August 10, 2026. A context window is still too small for a whole site, so the file stays small: a name, a short summary, and links to pages an agent can read.
Name the file llms.txt. Put it at the root, such as https://example.com/llms.txt, or under a path such as https://example.com/docs/llms.txt. A file covers the URLs under that path, and the more specific file wins when two apply. Posting Dude publishes one at /llms.txt. Markdown copies of pages use .md. rel="alternate" with type="text/markdown" points at a copy, and rel="describedby" points at the llms.txt for that path. Both can be a link element or a Link header. This checker reads the file you submit. It does not crawl the site for those tags.
How it differs from robots.txt and sitemap.xml
robots.txt and llms.txt answer different questions. robots.txt tells a crawler which paths the site asks it not to fetch. It does not describe the product. A Disallow does not edit llms.txt, and a link in llms.txt does not override a Disallow. Whether GPTBot, ClaudeBot, or PerplexityBot may fetch a path is a robots.txt question.
sitemap.xml is a list of URLs for search engines. The llms.txt spec contrasts the two directly. A sitemap often leaves out markdown versions of pages, it does not list helpful URLs on other hosts, and the full list is usually too large for a context window. llms.txt is meant to be read on demand, with the long pages behind the links. It is not crawl permission, and it is not a complete index.
The format
The sections go in a fixed order. A byte-order mark is optional. An H1 with the project or site name is the only required section. A blockquote holds the short summary. After that come paragraphs and lists, with no headings. Then come H2 file lists. Each item is a markdown link, [name](url), optionally followed by a colon and a note. An H2 titled Optional is the conventional place for links an agent can skip. A file that is only an H1 still follows the spec.
A missing summary, a relative URL, a repeated link, a very large file, or a link that does not end in .md is a suggestion. The proposal recommends those habits. It does not reject a file for skipping them.
# Northwind Docs > Northwind Docs is the API reference and guides for the Northwind library. Read the quick start before the reference. Pages linked below have markdown copies. ## Docs - [Quick start](https://example.com/docs/quickstart.md): Install, first request, and the project layout - [API reference](https://example.com/docs/reference.md): Every public function ## Optional - [Changelog](https://example.com/docs/changelog.md): Release notes an agent can skip
Common mistakes
One common miss is no file, while the host still returns HTTP 200 and an HTML page. Single-page apps do that for unknown paths. The result shows the status and the content-type. An HTML document fails. If the body is markdown and the content-type says HTML, that is a suggestion: the text parsed, and the label is wrong.
A file list fails when an item is not a link. A line like "- Pricing: plans and billing" names a page and does not link it. Each H2 item needs [name](url). A URL with a space, or a link that never closes, is malformed. Lists that appear before the first H2 are notes. They do not need links, and they are not checked as file-list items.
Headings other than the H1 and the H2 section titles are outside the format. Detail cannot contain headings. A second H1 fails, because the spec uses one H1 for the name. Keep explanations in paragraphs, and start each group of links with an H2.
A long file works against the proposal. The spec says to keep llms.txt small enough for a context window, and it sets no character cap. Token counts here are an estimate: characters divided by four, not a tokenizer. A suggestion appears around 80,000 characters. A URL fetch stops at 512 KB.
A proposal, not a ranking factor
llms.txt is a proposal, not a ranking factor. A file that passes this check does not, by itself, change search rankings or earn a citation in an assistant. Those products still choose what to fetch. robots.txt is the access file. A title and a meta description are the snippet text, when a search engine uses what you wrote.
Pasted text is checked in the browser and is not uploaded. A URL check is fetched on this server and is not stored. The link check requests up to 20 http and https URLs from the file lists, shows status codes, and refuses private and local addresses.
The AI crawler checker reads robots.txt for GPTBot, ClaudeBot, and the other named crawlers. The meta title checker previews a title and description before you publish. The free tools index lists the other tools on this site. Pricing lists Launch and Studio. The why page covers what happens when distribution stops after launch day.