What is llms.txt?
llms.txt is a proposed standard: a plain Markdown file placed at the root of a website (yoursite.com/llms.txt) that gives large language models and AI agents a short, curated guide to the site, including what it is about and links to its most useful pages. It was proposed in September 2024 by Jeremy Howard of Answer.AI.
- Last updated 14 October 2026
- Proposed in September 2024
- Lives at /llms.txt in Markdown
- Curates your most useful pages
- Complements robots.txt and sitemaps
In one line
llms.txt in one line
llms.txt is a short, human-readable map of your best content, written for AI systems rather than search crawlers.
How it works
How llms.txt works
- Step 1 Title and summaryStart with your site or brand name as a heading and a one-paragraph summary of what you do.
- Step 2 Key sectionsAdd Markdown sections such as Services, Guides and About, each listing links with one-line descriptions.
- Step 3 Optional detailsInclude an Optional section for lower-priority links that can be skipped when context is limited.
- Step 4 Publish at the rootUpload the file so it loads at yoursite.com/llms.txt, and keep it updated as pages change.
Example
An llms.txt example
# Sharma Dental Clinic > Family dental clinic in Salt Lake, Kolkata, offering check-ups, root canals, implants and braces since 2010. ## Treatments - [Root canal treatment](https://example.com/root-canal/): cost, steps and recovery - [Dental implants](https://example.com/implants/): who they suit and prices ## About - [Our dentists](https://example.com/team/): qualifications and experience - [Contact and timings](https://example.com/contact/) ## Optional - [Blog](https://example.com/blog/)
This fictional example shows the structure: a heading, a short summary in a blockquote, then grouped links with descriptions.
Why it matters
Why llms.txt matters
Clarity for AI agents
Agents browsing on a user’s behalf can find your most important pages quickly.
Control over the summary
You decide how your brand and offer are described at the top.
Low effort
It takes an hour to create and maintain, with no effect on human visitors.
Adoption
Does llms.txt actually help?
llms.txt is still a proposal rather than an official web standard, and major AI companies have not confirmed that they use it to rank or select sources. Some AI tools and documentation platforms do read it, and many developer and SaaS sites now publish one.
Our view: it is a low-cost, low-risk addition that can help AI agents understand your site, but it is not a substitute for crawlable pages, clear content and a strong reputation across the web. Treat it as one part of AI search readiness, alongside allowing AI crawlers in robots.txt and adding structured data.
Compared
llms.txt vs robots.txt vs sitemap
Three files at the root of your site, with different jobs.
| llms.txt | robots.txt | XML sitemap | |
|---|---|---|---|
| Purpose | Guide AI to your best content | Tell crawlers what they may access | List URLs for search engines |
| Format | Markdown | Plain text rules | XML |
| Audience | LLMs and AI agents | All crawlers, including AI bots | Search engine crawlers |
| Status | Proposed standard | Long-established standard | Long-established standard |
| Controls access? | No | Yes | No |
Checklist
How to add llms.txt to your website
- Write a one-paragraph summary of your business
- List 10–30 of your most useful pages with one-line descriptions
- Group links under clear headings
- Save as llms.txt and upload to your site root
- Check it loads at yoursite.com/llms.txt
- Make sure robots.txt does not block the AI crawlers you want
- Review the file whenever you add or retire key pages
Step by step
How to create an llms.txt file in WordPress or Shopify
WordPress: create the file in a text editor, then upload it to your site root with your hosting file manager or SFTP. Some SEO plugins can now generate it for you; check that the output lists your best pages, not every URL.
Shopify: Shopify does not allow arbitrary files at the root, so a common workaround is to create a page and redirect /llms.txt to it, or use an app that serves the file. Test the URL after setup.
Any platform: keep the file short, update it when you publish or retire key pages, and make sure your robots.txt allows the AI crawlers you want to read it.
What to include
What to include and what to leave out
Include your core service or product pages, pricing, key guides, about and contact pages, and policies customers often ask about. Leave out thin pages, tag archives, login areas and anything you would not want quoted. Write each link description as a plain one-line summary, because that text may be all an AI system reads before deciding whether to open the page.
Mistakes
Common llms.txt mistakes
- Listing every URL instead of curating the best pages
- Leaving out descriptions, so AI systems cannot tell pages apart
- Linking to pages blocked in robots.txt
- Forgetting to update the file after a redesign
- Expecting it to replace good content and structured data
Who should use it
Which websites should add llms.txt?
Developer documentation, SaaS products, publishers and any business with a lot of reference content benefit most, because AI agents are often asked to find specific answers in those sites. Local businesses and small service sites can add a short version in minutes; it costs nothing and gives AI tools a clear summary of who you are, what you offer and where to find prices and contact details.
Related terms
Related terms and next steps
FAQs
llms.txt: frequently asked questions
Straight answers to common questions.
Is llms.txt required?
Does llms.txt block AI crawlers?
What is llms-full.txt?
Will llms.txt improve my Google rankings?
Where should llms.txt be placed?
How long should an llms.txt file be?
Next step
Is your site ready for AI search?
Get a free check of AI crawler access, llms.txt and entity signals.
- Free, with no obligation to hire us
- A personal video walkthrough, not an auto-generated PDF
- Reply within one working day
Claim your free audit
We reply within one working day. Your details are never shared.