What llms.txt is
It is a convention proposed by Jeremy Howard (Answer.AI) and published in September 2024: a Markdown text file placed at /llms.txt that gives AI agents an overview of the site and links to content designed to be read by models.
Its rationale is practical. HTML pages carry menus, scripts and ads that make them hard to read, and models have a limited context window. A concise summary with well-chosen links makes it easier for them to find what matters. The proposal is meant to be used when someone asks about something, not for training.
What the file looks like inside
The format is simple and always follows this order:
A top-level heading with the name of the site or project. It is the only mandatory part.
A blockquote with a short summary of what the site is.
Optionally, a few paragraphs with more context.
Second-level sections with lists of links in Markdown format (the name in square brackets and the address in parentheses), with an optional note.
An “Optional” section for secondary links that the agent can skip if it is short on context.
The proposal adds that pages can offer clean Markdown versions (for example, page.html.md). That is a technical detail you do not need to get started.
What Google and others say about llms.txt
Google is blunt in its guide to AI features: you do not need to create machine-readable files, AI text files, markup or new Markdown to appear in Search. It adds that its Search ignores llms.txt files, so they neither help nor harm, and that there is no problem keeping them if they serve you for other services.
As for OpenAI, its crawler documentation does not mention it as a requirement for appearing in ChatGPT search; it only talks about allowing OAI-SearchBot. And the llms.txt page itself makes clear that it is an open proposal inviting community feedback, not a formal standard.
Beware of promises
If someone sells you llms.txt as the key to appearing in ChatGPT or Gemini, be wary. There is no official documentation to back that up.
When it makes sense to create one
It can add something when people read your website with AI tools on request: a product’s technical documentation, a long guide, a heavily structured catalogue. For a local business with a five-page website, the benefit is minimal.
- It may be worth it if your website has a lot of documentation and your users consult it with assistants.
- It is optional if you already have a clear website, with a page per service and good structure.
- It is not a priority if you still have basic indexing problems, thin content or contradictory details.
Before that, review what is documented: how to appear in Google’s AI answers and how to appear in ChatGPT.
What is documented and carries more weight
If your goal is for assistants and search engines to be able to read your website, there are clearer measures than llms.txt, backed by each company’s documentation.
- Allow the crawlers: OpenAI recommends allowing OAI-SearchBot in robots.txt, and Perplexity does the same with PerplexityBot. Changes take about 24 hours to apply.
- Indexable, useful pages: for Google’s AI features, the requirement is that the page is indexed and can be shown with a snippet.
- Clear, original content: direct answers, verifiable facts and a page per service.
- Measure in Search Console: it tells you whether Google indexes your pages and how many impressions you have in its AI features.
How to create one if you decide to
List the pages that best explain your business: services, frequently asked questions, contact and terms.
Write the file in a text editor, with the heading, the summary and the link sections, using full URLs.
Save it as llms.txt and upload it to the root of the domain, so it opens at yourwebsite.com/llms.txt.
Check that it loads without errors and is not blocked by your configuration.
Keep it up to date: an outdated llms.txt links to pages that no longer exist and undermines credibility.
On WordPress, some plugins generate it automatically. Use them with judgement and always review what they publish. It is a small piece within our GEO and AI visibility work, not the centre of it.
Frequently asked questions
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may or may not visit and is a long-established convention. llms.txt does not control access: it is a Markdown summary with links designed to help an AI agent understand your site. They serve different purposes and do not replace each other.
Does llms.txt improve my rankings on Google?
No. Google says its Search ignores these files, so they neither help nor harm your visibility in its results. If you decide to create one, do it for its other uses, not expecting an effect on SEO or on Google’s AI answers.
Do ChatGPT, Gemini or Perplexity read it?
No official documentation from OpenAI, Google or Perplexity presents it as a requirement for appearing. Some tools and agents do use it, but that does not mean it influences these assistants' answers. It is best not to assume it does.
Can having one hurt me?
Google says that keeping llms.txt files is perfectly acceptable and that it does not take them into account. The only practical risk is publishing outdated information or broken links, so if you create one, you will need to keep it up to date.
