llms.txt: What It Is, Does It Work, and Should You Add One?

llms.txt is a proposed Markdown file, placed at the root of a website (/llms.txt), that gives large language models and AI agents a short, curated summary of the site with links to its most useful pages. It is a community proposal published at llmstxt.org, not an official web standard, and Google says its Search and AI features don’t use it.

That gap between the hype and the documentation confuses many teams. This guide covers what the file is, what Google and the main AI vendors actually say, where it can still help, how to write one, and what matters more for your AI visibility.

Key takeaways

• llms.txt is a proposal from Jeremy Howard of Answer.AI, first published in September 2024, for a Markdown file that summarizes a website for AI systems.
• Google states that Google Search, including AI Overviews and AI Mode, ignores llms.txt, so adding one neither helps nor hurts you there.
• The crawler documentation of OpenAI, Anthropic and Perplexity does not describe llms.txt as an input for choosing which sources to cite.
• llms.txt does not control crawling or AI training. That is the job of robots.txt.
• The file is cheap to create and can help AI agents and developer tools, so adding one is reasonable if you don’t expect it to change AI answers on its own.

What is llms.txt?

Jeremy Howard, co-founder of Answer.AI, published the llms.txt proposal on September 3, 2024. The problem it addresses is practical: web pages are full of navigation, scripts and layout code, while a language model can only read a limited amount of text at once. An AI agent trying to understand your site benefits from a compact starting point.

According to the spec, the file contains, in this order:

1. An H1 with the site or project name. This is the only required element.
2. A blockquote with a short summary.
3. Optional free text with more detail, using any Markdown except headings.
4. H2 sections listing links, written as [name](url) with an optional note.
5. An “Optional” section for secondary links an agent can skip.

The proposal also suggests offering clean Markdown versions of important pages at the same URL with .md appended.

What about llms-full.txt?

llms-full.txt is not part of the llmstxt.org proposal. It’s a companion file some sites publish with their full content in one document, mostly for developer documentation. Anthropic’s and Perplexity’s documentation sites both publish one.

What Google says about llms.txt

Google’s guide to optimizing your website for generative AI features is unusually direct. In its list of things you can ignore, llms.txt comes first. You don’t need AI text files “to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.”

The guide adds that creating one for other services is “completely fine” and “will neither harm nor help your site’s visibility or rankings in Google Search.” It lists llms.txt alongside other myths such as “chunking” content and special schema markup for AI.

The Lighthouse audit

One Google tool does look at the file. In 2026, Chrome’s Lighthouse added an Agentic Browsing category with an llms.txt audit. Chrome’s documentation calls llms.txt “an emerging convention” and marks the audit Not Applicable when the file is missing, “as providing the file is optional at the moment.”

The audit flags server errors, and its published code checks for an H1, at least one Markdown link and a minimum length. So Lighthouse checks that your file is well-formed for agents. It isn’t a Search ranking signal, and a missing file doesn’t fail it.

Do ChatGPT, Claude and Perplexity use llms.txt?

None of them documents it. OpenAI’s crawler documentation explains how to manage OAI-SearchBot (search) and GPTBot (training) through robots.txt. Anthropic’s and Perplexity’s crawler pages also focus on robots.txt. None presents llms.txt as a signal for which sources get cited.

Two things often get confused:

• Publishing vs reading. OpenAI, Anthropic and Perplexity all publish llms.txt files for their own developer docs. That shows they find the format useful for documentation, not that their assistants read your file when answering a buyer.
• Fetching vs using. Some articles cite server logs showing an AI crawler requested /llms.txt. A request means the file was fetched, not that a model used it to write an answer.

Until a vendor documents otherwise, treat claims that an assistant “reads llms.txt for answers” as unverified.

Where llms.txt can still be useful

• Documentation and developer tools. When developers point coding assistants or agents at your docs, a clean index with Markdown pages saves them from parsing heavy HTML. This is where the format is most adopted.
• AI agents acting for users. Chrome’s audit page says that without the file, “agents may spend more time crawling the site to understand its high-level structure.”
• A clarity exercise. Summarizing what you sell, to whom and at what price is the same work that improves the pages assistants do read.

Skip it if nobody will maintain it. A file with old pricing or dead links is worse than none.

llms.txt vs robots.txt vs sitemap.xml

To allow or block specific AI crawlers, see our guide to AI crawlers and robots.txt.

How to write an llms.txt file: template and example

1. Start with an H1 and a blockquote. Your brand name, then one sentence on what you do and for whom.
2. Add plain facts. Category, target customer, starting price, key integrations. Statements, not taglines.
3. List your most useful pages under H2s. Product, pricing, comparisons and docs, each with a one-line note.
4. Move secondary pages to an “Optional” section.
5. Serve it at /llms.txt with a 200 status, then run the Lighthouse Agentic Browsing audit.
6. Generate it if you can. Citegram’s own site does this: citegram.com publishes /llms.txt and /llms-full.txt, generated at build time from its page and blog list.

Here is an example for a fictional B2B CRM:

# Northwind CRM

> Northwind CRM is a sales CRM for B2B startups with 5 to 50 people. Plans start at $15 per user per month.

Northwind CRM includes pipeline management, email sequences and a built-in dialer. It integrates with Gmail, Outlook and Slack.

## Product
- [Features](https://northwind.example/features): pipeline, email sequences, reporting
- [Pricing](https://northwind.example/pricing): plans, user limits, annual discount
- [Northwind vs Pipedrive](https://northwind.example/vs/pipedrive): features and pricing side by side

## Optional
- [Changelog](https://northwind.example/changelog)
- [About us](https://northwind.example/about)

What moves AI visibility more than llms.txt

If you want to be named when someone asks an assistant “best CRM for a 20-person startup”, the leverage is elsewhere:

• Crawlable, indexable pages. Assistants that search the web must be able to find and read your pages.
• Clear facts on the pages assistants read. We cover why in how AI assistants choose the sources they cite.
• Third-party coverage. Comparison questions often pull from review sites, roundups and forums.
• Structured data that matches the page. Google says no special schema is needed for AI features, but correct markup still supports rich results. See schema markup for AI search.
• Measuring answers. The broader discipline is covered in what generative engine optimization is and our guide to answer engine optimization.

Check whether llms.txt changes your AI answers with Citegram

If you add the file, compare answers before and after instead of assuming an effect. Citegram is a free Chrome extension built for that:

1. Add the prompts that matter. A handful of buyer questions where your brand should appear.
2. Run a baseline before you publish. Citegram asks each prompt in ChatGPT, Gemini, Perplexity, Claude, DeepSeek, Grok and Google AI Overviews in background tabs, using the accounts already signed in to your browser. No API key; questions count toward your own plan usage. First results arrive in about 10 minutes.
3. Record where you stand. For each answer, Citegram records whether your brand is mentioned, its position and the sources cited.
4. Publish the file and rerun the same prompts over the following weeks. Answers vary from run to run, so compare trends.
5. Compare share of voice and cited domains per brand and per assistant over time.

You can observe changes, but you can’t prove the file caused them. Answers also come from your own accounts, so personalization, memory and location can affect them.

FAQ

Does ChatGPT read llms.txt?

OpenAI has not documented ChatGPT using llms.txt to choose sources. Its crawler documentation explains how to control OAI-SearchBot and GPTBot through robots.txt and doesn’t present llms.txt as a signal. OpenAI does publish an llms.txt for its own developer docs, which is a different use.

Will llms.txt improve my Google rankings?

No. Google says Google Search ignores llms.txt files and that creating one “will neither harm nor help” your visibility or rankings. That includes AI Overviews and AI Mode.

Is llms.txt an official standard?

No. It’s a proposal published at llmstxt.org in September 2024 and refined with community input. It isn’t an IETF or W3C standard, and Chrome’s Lighthouse documentation calls it “an emerging convention.”

Can llms.txt block AI training?

No. llms.txt contains no directives and controls nothing. To limit crawling for AI training, use robots.txt rules for the relevant crawlers, such as OpenAI’s GPTBot or Google-Extended.

Test what actually changes your AI answers

Before and after any change, from llms.txt to a rewritten pricing page, the useful question is whether the answers moved. The free plan lets you track one prompt across all major assistants. Pro adds unlimited prompts for $10 a month. See pricing.