Almost half your docs traffic is AI, time to understand the agent experience

Right now, an agent is reading your docs.

A little software emissary, dispatched by a developer, a support tool, a sales workflow, or (increasingly) another agent entirely. It has a job: find the right information, extract what it needs, and bring back something useful.

It won't compliment your typography or linger on your landing page. It shows up with a task, finds what it came for, and leaves. And if it can't find what it needs, it doesn't file a bug report or send your team a frustrated email. It just moves on quietly to someone else's product or to a hallucinated answer it made up on its own.

That's the thing about agents: when they fail, you don't hear about it. There's no bounce rate alert, no angry support ticket. There's just a developer somewhere who got a wrong answer and will now blame your product for it, or move to a competitor entirely.

The journey of an agent

An agent’s journey typically begins the same way. It arrives on a domain and starts scanning for signals.

It needs to understand where to begin, which content is relevant, and what information applies to its specific task.

When those answers are clear and accurate, it continues navigating the site. When they aren't, it explores a few alternative paths and, if it still can't find clarity, moves on.

Agents are designed to move forward, not to puzzle things out for long stretches. Good documentation makes this forward motion easy.

What agents look for first

Before an agent reads a single paragraph, it looks for structure or a map.

On Mintlify sites, that map is llms.txt. A simple file that says, plainly, here is the content that matters, here is how it is organized, and here is what you can ignore.

This one detail changes the entire starting experience. Instead of guessing, the agent knows exactly where to go. Instead of crawling everything, it can focus on the parts that are meant to be understood.

Speaking clearly without saying everything twice

Humans and agents want the same things from documentation: clarity, consistency, and structure.

But they get there in different ways. For humans, visual design, narrative, and explanation help build mental models to navigate your documentation. These are abstractions of the files that agents rely on for structure. You can create content that is usable at both levels for humans and agents.

Content negotiation lets you serve the same underlying knowledge in different ways depending on who is asking. This means rich pages for browsers and clean, predictable representations for agents.

Nothing is hidden or duplicated and each reader gets what they need.

Time to consider not just human visitors, but to treat agents as first-class citizens.

• Cloudflare releases real-time content conversion to Markdown

When this works well, agents do not struggle to extract meaning and they can simply read and move on.

Seeing how agents actually use your docs

For a long time, agent traffic was invisible.

With Mintlify’s AI traffic analytics, you can see which agents are visiting your site, what pages they read, and where they stop. This changes how you think about documentation quality.

You start asking better questions.

• Are agents finding the right entry points?
• Are they missing important information in your documentation?
• Did it leave after one request or explore further?

These answers tell you whether your docs work in the environments where decisions are now being made.

When an agent can move through your docs and bring correct answers back to the person prompting it, you build trust and confidence with your users. Even when there is an agent mediating the interaction between your docs and the user. That’s how better answers, safer actions, and fewer hallucinations happen in practice.

Building for the readers you cannot see

Rather than just taking the place of human readers, agents expand what those readers can accomplish.

They read on behalf of users, tools, and workflows that depend on accurate documentation. If your site is easy for agents to understand, it becomes easier for everyone downstream.

We spend a lot of time thinking about this at Mintlify. About how agents discover content, how they decide what to trust, and what makes them keep reading instead of giving up.

Accurate documentation is no longer just something people visit. It’s something systems rely on.

That's the experience we are building for at Mintlify.