Good pages are only part of the job. Configuration files tell agents how to find your content and provide product context that no single page contains.
You’ll decide what to include, what to leave out, and which context agents can’t infer on their own.
llms.txt
llms.txt is an emerging standard, similar in spirit to robots.txt. llms.txt files give agents a curated, structured view of your content. Mintlify generates one automatically so you don’t have to start from scratch. For most sites, this is perfect. But you can replace it with a custom file for more control.
The generated llms.txt lists every page so agents can see the scope of your site without crawling it. If much of your content isn’t useful to agents, you can replace the generated file with a shorter custom version.
Make sure a custom file still gives agents enough context and follows the llms.txt specification. Otherwise, agents may miss useful information.
Here are some examples.
Weak llms.txt without sufficient context:
llms.txt that follows the standard and includes context:
llms.txt. However, if you are going to create a pared down version for agents, consider the following guidelines.
What to include:
- Your most important conceptual and how-to content for common tasks
- Pages that answer common questions
- Reference material that agents will frequently need to cite
- Changelog entries and release notes (high noise, low value for question-answering)
- Deprecated or legacy content you haven’t deleted yet (may confuse agents and lead to incorrect answers)
- Internal or pre-release content that isn’t ready to be surfaced (not yet relevant to tasks)
- Blog posts (useful for humans finding you; not useful for AI answering questions)
Excluding content from
llms.txt doesn’t hide it from agents. Agents can still find these pages by traversing your site.If content is irrelevant to agents, consider deindexing it or removing it from your site entirely.llms.txt from top to bottom. If an agent has a context limit and can only process part of the file, what comes first matters. Lead with your most important content then reference.
llms-full.txt
Some agents request llms-full.txt instead of llms.txt. The difference: llms-full.txt includes the full content of each page inline, rather than just links. More to work with, but a much larger file.
Mintlify generates this automatically. The same inclusion/exclusion logic applies. If you’re customizing llms.txt, consider whether llms-full.txt needs the same treatment.
Common mistakes with llms.txt files
Listing every page without descriptions. An agent can’t infer what a page covers from its title alone. “Advanced configuration” could mean anything. Add a brief description to every entry.
Putting reference content first. Most questions are conceptual or task-oriented. If your API reference is at the top and getting started content is buried, an agent will reach for the wrong content type.
No product description. The opening > block is what tells an agent what your product actually does. Without it, an agent approaching your site cold has no grounding context.
skill.md and SKILL.md
Mintlify automatically generates a root skill.md for public documentation sites. This capability summary tells agents what they can accomplish with your product, what inputs they need, and which constraints apply. It stays up to date as your documentation changes.
Use a custom root skill.md when you need to override the generated summary. For several installable skills, publish directories containing SKILL.md files. Mintlify exposes supported skill discovery endpoints so agents can find them.
A skill is a reusable capability that an agent can discover and invoke. Skills package step-by-step instructions that tell an agent how to complete a specific task with your product. Optional scripts and reference material can be included to help the agent complete the task.
If your product has an API and developers are building agents that use it, publishing skills on your site makes those capabilities directly available to agents. An agent working in a developer’s codebase can find your skill, read the instructions, and integrate with your API without the developer having to explain how.
A skill is a directory containing at minimum a SKILL.md file:
SKILL.md frontmatter requires a name and description. The body is plain markdown instructions for the agent:
llms.txt and skill.md may already cover the need. Read the Mintlify skill.md guide and Agent Skills specification before overriding them.
Structured sources
When your product has an API, keep its OpenAPI document or other machine-readable schema accurate. Structured sources give agents precise parameter types, required fields, enums, and response shapes that prose may omit. Treat the schema as the source of truth for API structure. Use hand-written pages for concepts, workflows, examples, and decisions the schema cannot explain. Avoid maintaining the same field definition independently in several places.CLAUDE.md
CLAUDE.md is read by Claude Code before it makes any changes to your repository. If your team uses Claude Code to help write or maintain content, this file is where you give those tools additional context.
A well-written CLAUDE.md for a content repository details:
Audience: Who your readers are, so Claude Code considers their needs when writing or editing content.
Terminology: The canonical terms for your product’s concepts, so coding agents don’t introduce inconsistent language.
Content types: What each type of page should do and shouldn’t do (tutorial, how-to, reference, explanation).
What not to do: Conventions your team has found Claude Code tends to violate. For example, “don’t add introductory paragraphs that restate the title” or “don’t use passive voice.”
.claude/CLAUDE.md
AGENTS.md
AGENTS.md serves the same purpose for your content repository as CLAUDE.md, but for a broader set of coding agents. Where CLAUDE.md is read specifically by Claude Code, AGENTS.md is read by OpenAI Codex and other coding agents that look for this file at the start of a session.
If your team uses multiple AI coding tools to maintain content, maintaining both covers more ground. The content can be the same—audience, terminology, content type rules, and style conventions—or AGENTS.md can be a simplified version of your CLAUDE.md.
Serve markdown to agents
Agents extract content more reliably from markdown than from HTML. HTML pages contain navigation, scripts, sidebars, and other markup agents have to work around. Markdown gives them just the content. Mintlify serves markdown versions of your pages at.md URLs automatically. For example, /your-page.md alongside /your-page. Most agents request markdown when it’s available. You don’t need to configure anything for this to work.
It’s worth pointing agents to this in your CLAUDE.md:
Page size
Agents have context limits. Pages that exceed roughly 50,000 characters get truncated when fetched. For most pages this isn’t a concern, but watch for pages that are too long:- API reference pages covering many endpoints
- Pages with large embedded code examples
- Mixed-topic pages that have grown over time without being split
What to decide before you configure
Before writing any of these files, answer three questions:- What content is most important for agents to have? Not all your content is equally valuable for question-answering. Identify your core content and make sure it’s prominent.
- What context do agents need that isn’t in the content? Audience, terminology, and conventions aren’t generally in your content, so agents don’t get them from reading your pages. Write them down.
- What should agents not surface? Deprecated content, internal notes, and pre-release material can confuse agents and mislead users. Be deliberate about exclusion.
You’ve configured the files that give agents the context they need to answer questions and perform tasks.Next up: Test whether agents can use your content — Build a repeatable evaluation set and catch regressions.