What is llms.txt and where does it live on a website?

llms.txt is a plain-text file placed at the root of a website (e.g. example.com/llms.txt) that lists the most important URLs and sections of the site, structured as Markdown for easy ingestion by language models. It was proposed in 2024 as a developer-experience convention for AI agents and ingestion pipelines, not as a Google ranking signal. The format is documented at llmstxt.org.

Does Google use llms.txt for ranking?

No. Google has stated through John Mueller and other public channels that llms.txt is not used as a ranking input for classical SERP results or AI Overviews. The signal for AI Overviews remains the underlying page content, schema, and authority of the site. Adding llms.txt does not move classical or AI Overviews rankings.

If it does not move rankings, why bother adding llms.txt?

Three reasons. First, AI agents and ingestion pipelines (Claude's web fetch, ChatGPT browsing, custom MCP servers, internal RAG systems) increasingly read llms.txt as a navigational hint. Second, it forces the operator to inventory and prioritize the site's important content, which is useful regardless of who reads it. Third, it is cheap — typically 2 to 4 hours to implement on a 50–500-page SMB site.

What is the difference between llms.txt and llms-full.txt?

llms.txt is the index — a Markdown list of URLs and short descriptions. llms-full.txt is the optional companion that includes the actual content of the indexed pages, inlined. For most SMB sites in 2026, llms.txt alone is enough. llms-full.txt is worth adding only for sites with stable, evergreen reference content that benefits from being available without crawl.

Should llms.txt include every page on the site?

No. The point is curation. Include the high-value pages: homepage, key service or product pages, the most authoritative blog content, the about and contact pages. Exclude low-value pages: tag archives, paginated listings, thin pages, legal boilerplate. A focused 20–80 link llms.txt is more useful than an exhaustive 500-link one.

Are there real AI agents that respect llms.txt today?

Yes, with caveats. Anthropic's Claude (via web fetch and computer use), some MCP server implementations, certain custom RAG pipelines, and a growing list of agent frameworks read llms.txt when it is present. Major search engines do not. The respect is opt-in by the agent, not enforced by any standard body.

Does llms.txt control whether AI assistants train on my content?

No. llms.txt is a navigational convention, not a permissions file. Training-opt-out for major AI providers is controlled through robots.txt directives (e.g. GPTBot, ClaudeBot, Google-Extended user agents) and through provider-specific settings. Do not confuse the two — adding llms.txt does not change what AI providers do or do not train on.

How often should llms.txt be updated?

Quarterly for most SMBs, immediately after a major site restructure or content launch. The file does not need to be regenerated on every blog post. The audience for llms.txt is agentic, not human, and agents tolerate moderate staleness — but a six-month-stale llms.txt that lists deleted URLs starts to actively mislead, which is worse than not having one.

Where does llms.txt fit in the broader GEO and AEO stack?

It is a low-effort hygiene item in the GEO layer, not a primary lever. The primary levers remain on-page passage quality, FAQ schema, internal linking, and authority signals. Build the substance first; add llms.txt as a closing-tier polish item once the foundation is in place.

Does llms.txt Help? What It Does and Does Not Do for B2B SMB Sites in 2026

Does llms.txt help?

Yes — in a narrow, specific way. llms.txt helps AI agents and LLM-powered tools navigate your site and find your most important content faster. It does not help Google rankings, does not affect AI Overviews citations, and does not control whether AI companies train on your content. For a B2B SMB site, it is worth a few hours to implement correctly and then update quarterly.

llms.txt is the file that everyone in SEO Twitter wanted to be the next robots.txt and that turned out to be something narrower. By 2026, the dust has settled enough to give B2B SMB operators a clear answer: it is worth adding, it is not a ranking lever, and the time investment is hours, not days. This post is the practical implementation guide for an SMB site that wants to ship llms.txt correctly and move on.

The file was proposed in 2024 by Jeremy Howard and documented at llmstxt.org. The idea is a curated, Markdown-formatted index of a site's most important content, served at the site root, intended for consumption by language models and AI agents that fetch the URL when navigating the site. The format is simple enough that a 50-page SMB site can ship it in an afternoon.

What llms.txt is not

Three claims I keep hearing and pushing back on:

It is not a Google ranking signal. Google has been explicit, repeatedly, through public channels including John Mueller's commentary on Bluesky and in Search Central, that llms.txt is not a ranking input for classical SERPs or for AI Overviews. The substantive signals for AI Overviews remain the same as for organic ranking: on-page content quality, schema markup, internal linking, and domain authority. Adding llms.txt does not move either surface.

It is not a training-opt-out mechanism. The file does not affect whether OpenAI, Anthropic, Google, or Meta train on the site's content. Training opt-out is governed through robots.txt directives targeting specific user agents (GPTBot, ClaudeBot, Google-Extended, anthropic-ai, ChatGPT-User, and similar). Confusing the two is the most common implementation error I see — operators adding llms.txt while leaving robots.txt unchanged and assuming they have controlled training. They have not.

It is not a substitute for schema markup. Schema (FAQPage, HowTo, Article, Service, Organization) drives AI Overviews and classical rich results. llms.txt is a navigational hint for agents that fetch the site. They serve different purposes and the schema work is the higher-leverage investment by an order of magnitude.

What llms.txt actually does

Three real functions in 2026:

First, it serves as a navigational hint for AI agents that fetch the site. Anthropic's Claude (via web fetch and computer use tools), some MCP server implementations, certain RAG-based assistants, and an increasing number of agent frameworks read llms.txt when it is present. The agent uses the file to decide which URLs are worth fetching for a given task. Without llms.txt, the agent has to fall back on sitemap.xml and on-page link discovery, which is slower and less reliable for surfacing the site's most important content.

Second, it forces a content prioritization exercise. Writing llms.txt requires deciding which 20 to 80 URLs on the site are worth promoting. For an SMB that has been publishing for two years without revisiting site architecture, this is a useful audit. The byproduct is often more valuable than the file itself — operators routinely find pages they had forgotten about and pages that should be deprecated.

Third, it is cheap signalling that the site is maintained and modern. The audience for that signal is small, but it includes some technical buyers and partners who check for it as part of due diligence. The downside is zero.

How to build llms.txt for an SMB site

The file is Markdown. The structure is roughly:

# Site Name

> One-paragraph description of what the site is and who it serves.

## Core pages

- [Homepage](https://example.com/): What we do.
- [Services](https://example.com/services): What we offer.
- [About](https://example.com/about): Who we are.
- [Contact](https://example.com/contact): How to reach us.

## Key content

- [Pillar post 1](https://example.com/blog/post-1): One-line description.
- [Pillar post 2](https://example.com/blog/post-2): One-line description.

## Optional

- [Older content index](https://example.com/blog/): Full blog archive.

Save the file at the site root as llms.txt. Serve it with content-type text/plain or text/markdown. Do not require authentication. Do not redirect.

The companion file llms-full.txt contains the actual content of the indexed pages, inlined. For most SMB sites, this is overkill and adds a maintenance burden. Add it only if the site has a small, stable set of reference pages that benefit from being available without crawl — for example, a documentation site or a product comparison hub.

What to include and what to exclude

Include: homepage, primary service or product pages, the about and contact pages, the 5 to 15 highest-authority blog posts, any pillar content hubs, and key trust pages (case studies, results, pricing if public). Exclude: tag and category archives, paginated listings, thin pages, search results, legal boilerplate, login pages, and anything you would not link to from the homepage.

A focused llms.txt of 20 to 80 links beats an exhaustive 500-link one. The agents reading the file are doing prioritization. Help them.

Where llms.txt fits in the broader GEO stack

For SMBs in 2026, the order of operations for AI search visibility is roughly:

On-page content quality (passage specificity, entity density, named tools and benchmarks).
Schema markup (FAQPage, Article, Service, Organization, HowTo where applicable).
Internal linking (hub-and-spoke architecture, descriptive anchor text, breadth of internal references to key pages).
External authority signals (digital PR, mentions in authoritative sources, citation in industry publications).
Technical hygiene (sitemap.xml, robots.txt, server-side rendering for AI crawlers, llms.txt).

llms.txt belongs in the bottom tier — useful, cheap, low-priority. If a B2B SMB is debating between writing llms.txt and adding FAQPage schema to its top 10 pages, FAQPage schema wins every time. The investment ratio is not close.

Common mistakes I see in SMB implementations

Three to watch for:

Listing every page. A 400-link llms.txt for a 500-page site signals nothing. Curate. The agents reading the file will not consume all 400 entries; they will use the first 20 to 50.

Stale links. A six-month-old llms.txt that lists deleted pages or redirected URLs actively misleads agents. Refresh quarterly. Set a calendar reminder.

Confusing llms.txt with crawler controls. Adding llms.txt while leaving GPTBot and ClaudeBot fully allowed in robots.txt is fine if that is the intent. Adding llms.txt and assuming it limits training is not. Decide on the training position separately, in robots.txt, with the appropriate user-agent directives.

llms.txt is hygiene — the real GEO work is on-page substance and schema

llms.txt is hygiene, not strategy. Add it in an afternoon, audit your top URLs while you do it, and move on. The actual GEO and AEO levers — on-page substance, FAQ schema, internal linking, authority — are where the multi-month investments belong. For the broader playbook on how an SMB site earns AI citations in 2026, the AI citation tracking post covers the substantive layer, the services overview covers what gets reviewed and refreshed monthly on engagement, and the GEO for SMBs guide covers the broader playbook.

The 10-minute llms.txt audit checklist

If you already have an llms.txt file — or just shipped one — run this checklist before moving on. Each item takes under two minutes to verify.

Does the file load at the correct URL? Visit yourdomain.com/llms.txt in a browser. If it returns a 404 or a redirect, the file is not serving. Fix the path or the server configuration before anything else.
Is the content type correct? Use curl -I yourdomain.com/llms.txt and check the Content-Type header. It should return text/plain or text/markdown. If it returns text/html, the file is being intercepted and rendered as a page rather than served as text.
Does the opening description actually describe the business? The first paragraph after the title is what an AI agent reads to decide whether the site is relevant to the query. "We help B2B SMBs in Vancouver grow with AI-powered marketing systems" is useful. "A leading provider of professional services" is not.
Are all listed URLs returning 200? Open five to ten of the linked pages. If any are 404s, redirects, or login-gated, remove them from the file. A broken link in llms.txt misleads the agent and wastes the crawl budget on that entry.
Does the file list fewer than 100 URLs? If your llms.txt has more than 80 to 100 entries, it is almost certainly including pages that do not need to be there. Tag pages, paginated archives, legal boilerplate, and search results should not appear. Trim to the 20 to 60 highest-priority URLs.
Is there a description for each key page? Each listed URL should have a one-line description after the link. Bare URLs with no context give the agent no signal about what the page covers. Add a plain-language description: [AI Marketing for SMBs](https://example.com/services): What we do, who we work with, and how to get started.
Is the file dated or does it have a last-modified note? AI agents and ingestion pipelines sometimes use the file modification date to decide whether to re-index. Set a calendar reminder to update the file quarterly, even if only one URL changes. A 12-month-old llms.txt with deleted pages is worse than no file.
Is the robots.txt consistent with the llms.txt intent? If llms.txt invites agents to index the site, check that robots.txt does not block GPTBot, ClaudeBot, or PerplexityBot. Conflicting signals confuse responsible crawlers. Decide on the training and indexing policy, then make both files consistent.

Eight checks, ten minutes. If all eight pass, the file is doing its job. If any fail, the fix is usually under five minutes. Do the audit once on publication and once per quarter after that.