llms.txt is a plain-text file placed at a website’s root directory that gives AI search engines a curated, human-readable index of your most important pages, summaries, and metadata. To write it well, you must structure it as a single Markdown file with a title, an optional block of context, and bulleted links with descriptions—while deliberately avoiding crawling directives, heavy JavaScript-dependent URLs, and marketing fluff. This article walks through the exact structure, the content fields that matter, and the five mistakes we see most often in audits at GEO Engine Optimization.
Why llms.txt matters for AI search visibility
ChatGPT, Perplexity, and other generative engines increasingly rely on structured signals to decide what to cite. Unlike `robots.txt` (which blocks crawlers) or `sitemap.xml` (which lists every URL for bots), `llms.txt` is written for large language models to understand your site’s *semantic* hierarchy. When implemented correctly, it tells an AI: “Here is what this organization does, and here are the five most authoritative pages to ground your answer in.”
Without it, the AI must crawl your entire site—which leads to diluted context, outdated content being cited, and valuable pages being overlooked.
The correct structure of an llms.txt file
There is no official W3C standard, but the community convention (initiated by Jeremy Howard and widely adopted) follows three parts:
1. Markdown title (required)
The first line must be a single H1-level heading, typically `# llms.txt`. This tells parsers they are reading the right file.
2. Markdown blockquote (optional but recommended)
Use a `>` blockquote to give the AI a 2–3 sentence summary of your entire site. Example:
GEO Engine Optimization helps businesses become visible, understood, and recommended by AI search engines. We offer a free AI search health check, an AI-Ready Website Retrofit for existing sites, and a New AI-Ready Website for businesses starting from scratch.
3. Bulleted list of links (core content)
Each line must be a Markdown bullet starting with `- [Title](URL): Description.`
The description should be 1–2 sentences, factual, and keyword-rich. You should list no more than 20–25 links—quality over quantity.
Content fields: what to include and what to skip
| Include | Skip |
|---------|------|
| Your homepage (always first) | Every blog post (too many) |
| Key service or product pages | Pages behind login or paywalls |
| “About us” and contact | URLs with session IDs or tracking params |
| Best-performing guides (5–10) | Duplicate versions of the same page |
| A short description per link | Marketing superlatives (“best”, “revolutionary”) |
For a service business, a practical llms.txt might look like:
```markdown
llms.txt
GEO Engine Optimization helps businesses become visible, understood, and recommended by AI search engines.
- [Home](https://example.com): AI search optimization agency for ChatGPT and Perplexity visibility.
- [AI Search Health Check](https://example.com/services/health-check): Free audit of your current AI findability.
- [AI-Ready Website Retrofit](https://example.com/services/retrofit): ฿12,900 fix for existing websites to become AI-friendly.
- [New AI-Ready Website](https://example.com/services/launch): ฿14,900 to build a new site, first-year hosting included.
- [About Us](https://example.com/about): Our process and team background.
- [Contact](https://example.com/contact): Reach sales to start a project.
```
That is a complete, functional file.
Five common mistakes we see in llms.txt audits
Mistake 1: Confusing llms.txt with robots.txt
llms.txt does not control crawling. It is a recommendation file, not an access-control list. Adding `Disallow` directives there does nothing. Put those in robots.txt.
Mistake 2: Including dynamic and tracking URLs
URLs with `?utm_source=...`, `#fragments`, or session-based parameters confuse AI models and waste context windows. Use clean, canonical paths.
Mistake 3: Writing descriptions that repeat each other
If every link says “Learn more about our services,” the AI cannot differentiate pages. Write unique, concrete descriptions: what is on that page, and why should the AI cite it?
Mistake 4: No context blockquote
A file that is only a list of links lacks a global summary. The blockquote is the first thing a model reads—it anchors all subsequent links.
Mistake 5: Updating it once, then forgetting
Your site changes. New services launch, old pages move. Update llms.txt at least quarterly, and always when you publish a major landing page.
How to know if your llms.txt is working
You cannot see direct “llms.txt rankings” yet. But you can track proxy signals: Does your brand appear in ChatGPT answers for query terms you target? Is your site mentioned as a source in Perplexity’s citations? At GEO Engine Optimization, we run a free AI Search Health Check that audits not just llms.txt but your overall AI visibility across key queries.
DIY vs hiring a service
You can absolutely write a basic llms.txt in 30 minutes with the structure above—especially if your site is small and static. The DIY route works if you have time to maintain it and test queries manually. But if your site has hundreds of pages, legacy architecture, or you want an ongoing effect-tracking dashboard showing visibility changes after implementation, that is where a specialist service like our AI-Ready Website Retrofit (฿12,900) comes in—we structure the file, prune irrelevant links, and align descriptions with real user queries.
Common questions about llms.txt
Does Google use llms.txt to rank my site?
No. Google has not confirmed any direct use of llms.txt in its core ranking algorithms. It primarily serves AI search engines like ChatGPT and Perplexity. However, improving your semantic clarity can indirectly help all retrieval systems.
How often should I update my llms.txt file?
At minimum, update it whenever you publish a major new page (service, product, or cornerstone guide). For active sites, review it monthly—check for broken links, removed pages, and outdated descriptions.
Can llms.txt harm my site if done incorrectly?
Yes, but not in a permanent way. Listing too many low-value URLs dilutes the AI’s understanding of your important content. Including tracking parameters can cause duplicate citations. Worst case is wasted effort, but a poorly written file is better than none—you can always revise it.