Claude Code Asks for Markdown 76% of the Time. ChatGPT Never Does.

Claude Code requested Markdown on 76% of its page fetches. ChatGPT-User, which produced 73% of all agent traffic against the same site over the same two months, requested Markdown on 0.1% of its fetches. Identical content, identical URLs, identical window. The entire difference lives in one HTTP request header that has been in the spec since 2001.

Rita Klubochkina, a senior frontend engineer at Evil Martians, and Travis Turner, the company’s tech editor, instrumented their own site from early May to early July 2026 and published the per-agent breakdown. We have argued before that llms.txt is ignored and that coding agents, not answer engines, are the real readers of the web. That post had the aggregate. This one supplies the mechanism it was missing.

One caveat to carry through the whole piece: their logs recorded roughly 268,000 agent requests against roughly 107,000 human pageviews, and the authors are explicit that this is an upper bound. Agent requests are counted server-side, where every asset fetch and retry shows up. Human pageviews are counted client-side, where blockers and bounced sessions vanish. The two numbers are not measured with the same instrument, so treat the ratio as a ceiling rather than a finding.

The volumes are local. The format split travels.

Here is the shape of the traffic, cleaned of generic indexers:

Client	Requests	Markdown	Markdown share
ChatGPT-User	196,973	272	0.1%
Claude Code	23,300	17,814	76%
Perplexity	7,728	~0	~0%
OAI-SearchBot	7,255	1,915	26%
GPTBot	3,579	1,104	31%

Markdown accounted for about 15% of agent reads overall, roughly 40,000 requests against roughly 227,000 for HTML. That 15% is a cleaned figure. The raw number is 21% with generic indexers included, and a tighter filter brings it down to 13%. Claude Code alone drove 43% of all legitimate Markdown traffic.

Do not reuse those per-client volumes as proportions for your own property. This is one site, with a technical audience, and its traffic mix reflects that audience. What transfers is the split itself: some clients declare a format preference and honor it consistently, and others never declare anything. A single blended “AI traffic” number hides that completely, which is why teams tracking one aggregate line have nothing to act on.

The mechanism is Accept, and it needs Vary

Content negotiation is the oldest unglamorous part of HTTP. The client states what representations it can use, the server picks one. Agents that ask for text/markdown are asking for the version of your page without the navigation, the cookie banner, the newsletter modal, and the 40KB of layout markup wrapped around 800 words of prose.

Two things have to ship together. A parallel .md route for every page, and a rewrite triggered by the request header:

# Detect a Markdown-capable client
map $http_accept $wants_markdown {
    default        0;
    ~text/markdown 1;
}

server {
    location / {
        # Same URL, different representation
        if ($wants_markdown) {
            rewrite ^/(.*)$ /$1.md last;
        }

        # Without this, a shared cache will serve the Markdown
        # body to the next browser that asks for the same URL
        add_header Vary Accept;
    }
}

The Vary: Accept line is the one people forget, and forgetting it is worse than not shipping the feature at all. Any cache between you and the client (your CDN, a corporate proxy, the browser itself) keys responses by URL. If you return two different bodies for one URL without declaring which request header selected them, the cache will hand a raw Markdown document to a human visitor, or hand HTML to the agent that asked for Markdown. The header is what makes the negotiation cacheable and correct.

Everything else is content work. Strip the chrome, keep the headings, keep the code blocks, keep the links as real links. The .md twin is the same article with the furniture removed.

The authors retired their own recommendation

Evil Martians had previously recommended a hidden div carrying an AI hint, a small instruction block meant to tell agents that a Markdown version existed. Across 268,000 requests they could not attribute a single Markdown fetch to it. Their own words: “we cannot attribute a single Markdown fetch to the hidden hint.” Zero tagged fetches.

The llms.txt numbers landed in the same place. /llms.txt was fetched about 660 times, /llms-full.txt about 110. Of roughly 770 direct fetches, only 37 came from a named AI assistant. The remaining 95% were crawlers, generic bots, and scripts. The referrer data was worse: of 117 hits, 106 came from a single client sending a frozen Chrome/111.0 user agent, which means more than 90% of the apparent interest was one misconfigured thing looping. Google’s John Mueller, quoted from Bluesky, put the ceiling on it: “no AI system currently uses llms.txt.”

That is the credibility beat in the whole study. The authors published a measurement that killed their own prior advice, and then published the correction. Publishing a recommendation is easy. Instrumenting it for two months and retracting it is the part almost nobody does.

Classifying the traffic is harder than serving it

The Accept plumbing is an afternoon of work. Knowing who is on the other end is the real engineering problem, and the authors say so directly: “classifying LLM traffic is the actual hard part.” Also: “there’s really no such thing as AI traffic.”

They landed on an edge classifier with five kinds, built by exclusion rather than by allowlist: ai, browser, crawler, scanner, library. Nothing gets tracked as agent activity unless the kind resolves to ai, or Markdown was actually served and the client is not noise. An allowlist of known user agents fails on contact with reality, because new clients appear weekly and half of them ship a generic or spoofed UA. Excluding what you can positively identify as a browser, a scanner, a scraper, or an HTTP library leaves a residue that is closer to the truth than any list of names you maintain by hand.

Two operational details are worth stealing. First, mine your .md 404s. As the authors put it, “your .md 404s are free signal about the titles agents expect you to have.” An agent guessing at agent-observability.md is telling you the slug it inferred from a citation somewhere. Second, register and 301 your TLD variants along with the raw origin IP. Hallucinated requests hit .com when you own .io, and if that hostname does not resolve to your infrastructure you never see the request at all.

Ship this in one afternoon

Pick your highest-traffic content section, not the whole site. Then:

1. Generate a .md twin for every page in that section, served at the same path plus the extension.
2. Add the Accept: text/markdown rewrite and add_header Vary Accept. Verify with curl -H 'Accept: text/markdown' and confirm the Vary header comes back.
3. Log the requested representation next to the classified client kind. One column for format, one for kind. That single pairing is what produced the 76% versus 0.1% finding.
4. Delete any hidden AI-hint markup you added on someone’s advice, including ours if we ever gave it. It measured zero.
5. Set a .md 404 report to land in your inbox weekly and read it as a list of titles agents believe you have.

The measurable claim to test on your own property is narrow: whether the clients reaching your content declare a format preference, and whether you honor it. Answering that takes a header, a route, and a log column. Everything past that point is optimization on a surface you have not built yet.

---

This analysis synthesizes Which AI actually reads your site? Two months of LLM traffic, measured by Rita Klubochkina and Travis Turner (Evil Martians, July 2026), and builds on our earlier work on the agent-readable surface, machine readability as a marketing KPI, and agents eclipsing human traffic as the new perimeter.

Victorino Group helps engineering and marketing teams instrument agent traffic and build the negotiated Markdown surface their content already deserves. Let’s talk.