llms.txt in Practice: What It Does and How We Build It

Few GEO topics are discussed as heatedly as llms.txt – and for few is the gap between promises and evidence as wide. We run the file on our own website, recently fully generated instead of hand-maintained. This article gives an honest assessment of what llms.txt can and cannot do – and shows the setup that keeps the file current by itself.

What llms.txt is – and wants to be

The proposal comes from Jeremy Howard (Answer.AI, 2024): a markdown file at /llms.txt that gives language models what they otherwise have to piece together from a website – who writes here, what is offered, where the important content lives. Plain text instead of HTML noise, curated instead of crawled.

The analogy to robots.txt is tempting but misleading: robots.txt is an established standard that every relevant crawler respects. llms.txt is a community proposal whose consumer side remains thin to this day.

The honest state of the evidence

In its favour:

• The effort is minimal – one text file in the root, no risk to anything existing.
• Adoption is growing, especially in developer documentation (AI tooling vendors, framework docs). In that niche, the format has become a de-facto convention.
• AI crawler hits on the file show up in server logs – occasionally, but genuinely.

Against it:

• No major AI provider has confirmed systematically consuming llms.txt for answers.
• Google has been dismissive – for classic search, the file plays no role.
• A measurable citation effect is not cleanly documented anywhere – including on our own domain.

Our conclusion: llms.txt is a bet with a tiny stake. If you place it, do so consciously as an experiment – and keep the effort correspondingly small. Which brings us to the actual point.

The real problem: the file goes stale

Most llms.txt files you find in the wild are snapshots: written once, never touched again. New articles are missing, old prices remain, moved pages point nowhere. A stale facts file is worse than none – it actively tells AI systems wrong things, and nobody notices because the file never shows up in a browser.

So the same principle applies to llms.txt as to sitemaps and structured data: derived, not maintained. The file must be built from the same sources as the website itself.

Our setup: llms.txt as a build artefact

Our website is a static Astro site with content collections. The llms.txt is rebuilt on every deploy, from three sources:

1. Curated header: key facts (company, founding, registry, way of working) and the most important pages – the rarely-changing part, kept as a template.
2. Central pricing source: every public price anchor comes from the same file that feeds the pages and the schema markup. If a price changes, it changes everywhere – including the llms.txt.
3. Content collections: every published article lands in the file automatically with URL, date and description; every glossary term with its detail-page URL. Scheduled articles appear on their publication day by themselves, because the same filter applies as for the sitemap and listing pages.

Technically this is an Astro endpoint (src/pages/llms.txt.ts) that renders a text file at build time – no server, no cron job, no maintenance. For bilingual projects with separate domains, each domain needs its own language version; that is the one pitfall when the deployment splits languages out of a single build.

You can see the result live: codeaeffchen.com/llms.txt.

What belongs in – and what doesn’t

In goes what an AI system needs for a correct recommendation:

• Who: company name, legal form, location, since when, registry data
• What: services in one line per offering, public prices
• Where: the most important pages with URL and one-sentence description
• Content: a complete article and glossary list with URLs

Out stay marketing superlatives, keyword lists, and claims that don’t appear on the website. AI systems cross-check sources – an llms.txt that promises more than the website delivers damages the entity’s consistency instead of helping it. Same principle as our facts page for AI systems: identical facts everywhere.

If you don’t have a build pipeline, the generator takes care of the format – maintenance then stays manual, so put a reminder in your calendar:

Also available as a standalone page with all guides: llms.txt generator.

Where it sits in the GEO toolbox

llms.txt is a small building block in a bigger picture. Based on everything we measure on our own domain, other levers work harder: glossary detail pages with clean schema, FAQ blocks with directly quotable answers, a consistent entity across all channels – our article on visibility in AI search gives the overview. llms.txt adds a machine-friendly front door on top, nothing more.

If you want to tackle this for your own website: the right moment is when the foundation is being worked on anyway – during a relaunch or a migration to a static stack. Then the generated version costs an hour instead of an afternoon. What such a foundation looks like is on our Astro services page.