Blog · Sep 8, 2026
The llms.txt guide
What it is
llms.txt is a proposed standard (by Jeremy Howard of Answer.AI): a Markdown file at your site root that summarizes what your site is about, in plain text AI systems can ingest directly. Think of it as robots.txt's helpful sibling — where robots.txt says what crawlers may do, llms.txt tells them what's worth reading.
Why it matters
AI crawlers have limited budgets. A large site might have thousands of pages; the crawler won't read them all. llms.txt hands the crawler a curated summary: who you are, what you offer, and links to the pages that matter. It's the difference between the AI skimming your nav menu and reading your pitch.
The format
Plain Markdown. Start with an H1 (your site name), a blockquote-style summary, then H2 sections. Link generously — every claim should point at the page that backs it up. Keep it factual; this is the text AI will quote.
A full example
# Acme Plumbing
> Acme Plumbing is a residential plumbing company serving
> Pittsburgh, PA since 1998. Licensed and insured (PA #12345).
## Services
- [Drain cleaning](https://acme.example/drain-cleaning): flat-rate,
same-day service within city limits.
- [Water heaters](https://acme.example/water-heaters): repair and
replacement, 6-year warranty on installs.
- [Emergency repairs](https://acme.example/emergency): 24/7,
$95 diagnostic fee waived with repair.
## Service area
Pittsburgh, Mt. Lebanon, Dormont, Bethel Park, and Peters Township.
## Contact
- Phone: (412) 555-0140
- [Request a quote](https://acme.example/quote)Where to put it
At the root: https://yoursite.com/llms.txt. That's where AI systems look. Serve it as text/plain or text/markdown. Don't block it in robots.txt — that defeats the purpose.
Common mistakes
- Marketing copy instead of facts. "World-class solutions" tells the AI nothing. Prices, areas, hours, and license numbers tell it everything.
- Stale information. If your prices change, update llms.txt the same day. AI quoting last year's prices is worse than quoting nothing.
- No links. A summary with no links is a dead end. Every service, location, and claim should link to its source page.
- Burying it.
/docs/llms.txtor/assets/llms.txtwon't be found. Root or nothing.
How CrawlReady checks it
Every scan fetches your /llms.txt, validates that it exists, parses the Markdown structure, and flags missing sections — no H1, no links, no contact info. If you don't have one, the report includes a starter file generated from your site's actual content.