Good documentation pages are necessary, but not sufficient. The files and configurations in this lesson determine what agents know about your product and how they find information in your docs, context that doesn’t come from reading individual pages. Decide what to include, what to exclude, and what agents need to know that they can’t infer from your content alone.Documentation Index
Fetch the complete documentation index at: https://learn.mintlify.com/llms.txt
Use this file to discover all available pages before exploring further.
llms.txt
llms.txt is an emerging standard, similar in spirit to robots.txt. llms.txt files give agents a curated, structured view of your content. Mintlify generates one automatically so you don’t have to start from scratch. For most sites, this is perfect. But you can replace it with a custom file for more control.
The automatically generated llms.txt lists every page in your documentation so that agents can see the full scope of your content without traversing your entire site. However, sometimes you want to expose less information to agents. If you have a lot of content that is not relevant to agents, you can create a custom llms.txt file to exclude it.
Be careful though, you want to make sure your custom llms.txt file still includes enough context for agents and follows the llms.txt specification. If you don’t, agents may not be able to find the information they need.
Here are some examples.
Weak llms.txt without sufficient context:
llms.txt that follows the standard and includes context:
llms.txt. However, if you are going to create a pared down version for agents, consider the following guidelines.
What to include:
- Your most important conceptual and how-to content that teach how to do frequent tasks with your product
- Pages that answer common questions
- Reference material that agent will be asked to cite frequently
- Changelog entries and release notes (high noise, low value for question-answering)
- Deprecated or legacy content you haven’t deleted yet (may confuse agents and lead to incorrect answers)
- Internal or pre-release content that isn’t ready to be surfaced (not yet relevant to tasks)
- Blog posts (useful for humans finding you; not useful for AI answering questions)
Excluding content from
llms.txt doesn’t hide it from agents. Agents can still find these pages by traversing your site.If content is irrelevant to agents, consider deindexing it or removing it from your site entirely.llms.txt from top to bottom. If an agent has a context limit and can only process part of the file, what comes first matters. Lead with your most important content then reference.
llms-full.txt
Some agents request llms-full.txt instead of llms.txt. The difference: llms-full.txt includes the full content of each page inline, rather than just links. More to work with, but a much larger file.
Mintlify generates this automatically. The same inclusion/exclusion logic applies. If you’re customizing llms.txt, consider whether llms-full.txt needs the same treatment.
Common mistakes with llms.txt files
Listing every page without descriptions. An agent can’t infer what a page covers from its title alone. “Advanced configuration” could mean anything. Add a brief description to every entry.
Putting reference docs first. Most questions are conceptual or task-oriented. If your API reference is at the top and getting started content is buried, an agent will reach for the wrong content type.
No product description. The opening > block is what tells an agent what your product actually does. Without it, an agent approaching your docs cold has no grounding context.
SKILL.md
A skill is a reusable capability that an agent can discover and invoke. Skills package step-by-step instructions that tell an agent how to complete a specific task with your product. Optional scripts and reference material can be included to help the agent complete the task.
If your product has an API and developers are building agents that use it, publishing skills in your documentation makes those capabilities directly available to agents. An agent working in a developer’s codebase can find your skill, read the instructions, and integrate with your API without the developer having to explain how.
A skill is a directory containing at minimum a SKILL.md file:
SKILL.md frontmatter requires a name and description. The body is plain markdown instructions for the agent:
llms.txt and CLAUDE.md are more relevant. Read the Agent Skills specification for the full format.
CLAUDE.md
CLAUDE.md is read by Claude Code before it makes any changes to your repository. If your team uses Claude Code to help write or maintain documentation, this file is where you give those tools additional context.
A well-written CLAUDE.md file for a documentation repository details:
Audience: Who your readers are, so Claude Code considers their needs when writing or editing content.
Terminology: The canonical terms for your product’s concepts, so coding agents don’t introduce inconsistent language.
Content types: What each type of page should do and shouldn’t do (tutorial, how-to, reference, explanation).
What not to do: Conventions your team has found Claude Code tends to violate. For example, “don’t add introductory paragraphs that restate the title” or “don’t use passive voice.”
.claude/CLAUDE.md
AGENTS.md
AGENTS.md serves the same purpose for your documentation repository as CLAUDE.md, but for a broader set of coding agents. Where CLAUDE.md is read specifically by Claude Code, AGENTS.md is read by OpenAI Codex and other coding agents that look for this file at the start of a session.
If your team uses multiple AI coding tools to maintain documentation, maintaining both covers more ground. The content can be the same—audience, terminology, content type rules, and style conventions—or AGENTS.md can be a simplified version of your CLAUDE.md.
Serve markdown to agents
Agents extract content more reliably from markdown than from HTML. HTML pages contain navigation, scripts, sidebars, and other markup agents have to work around. Markdown gives them just the content. Mintlify serves markdown versions of your pages at.md URLs automatically. For example, /your-page.md alongside /your-page. Most agents request markdown when it’s available. You don’t need to configure anything for this to work.
It’s worth pointing agents to this in your CLAUDE.md:
Page size
Agents have context limits. Pages that exceed roughly 50,000 characters get truncated when fetched. For most documentation pages this isn’t a concern, but watch for pages that are too long:- API reference pages covering many endpoints
- Pages with large embedded code examples
- Mixed-topic pages that have grown over time without being split
What to decide before you configure
Before writing any of these files, answer three questions:- What content is most important for agents to have? Not all your documentation is equally valuable for question-answering. Identify your core content and make sure it’s prominent.
- What context do agents need that isn’t in the content? Audience, terminology, and conventions aren’t generally in your documentation, so agents don’t get them from reading your pages. Write them down.
- What should agents not surface? Deprecated content, internal notes, and pre-release material can confuse agents and mislead users. Be deliberate about exclusion.
You’ve configured the files that give agents the context they need to answer questions and perform tasks.Next up: Keep content agent-friendly over time — How to maintain these properties as your product and team evolve.