'We Monitor Our Agents' Now Has an Outside Number: 75 to 94% Eligible, for One OpenAI Traffic Class

Roughly 75 to 94 percent. That is how much of OpenAI’s agentic internal Responses API traffic, from models at least as strong as GPT-5.5, was eligible for monitoring, according to an external red-team of the company’s internal agent monitoring systems. The authors attach a warning to the figure: “substantial uncertainty in both directions.”

The write-up was published by METR, which says the work involved one METR employee and one Redwood Research employee. OpenAI had put a number on its monitoring before. In March, its own report gave a near-total coverage figure for agent traffic, which we cited at the time. METR’s range is a different kind of number. It comes from outside, it measures eligibility for one stated slice of traffic, and it arrives with error bars and a list of holes. The two figures use different scopes and definitions, so neither refutes the other. Once coverage has been stated with a scope and a bypass list, a bare “we monitor our agents” starts to sound like what it is: a claim nobody has shown a measurement for.

Two Numbers Hiding in One Word

The figure METR reports is eligibility. Eligible traffic is traffic the monitoring system is supposed to see. Whether the monitor actually ran on it is a second question, and the report keeps the two apart.

The clearest example is internal Codex. For internal Codex agents, 99.2% of traffic was eligible for monitoring. Yet compaction, the step that condenses a long agent context, likely left “on the order of 10% of eligible internal Codex traffic not being monitored between May and July 2026.” OpenAI had found and fixed that problem itself.

So the same population of agents produced a near-total eligibility figure and, inside it, a likely unmonitored share on the order of 10%. Neither number is wrong. They answer different questions. A dashboard that shows only the first one would have looked excellent for the whole period in which the second one was a problem.

This is the distinction any buyer can now ask for. Eligible is a design property: which traffic the system was built to cover. Monitored is an operational property: which traffic the monitor actually evaluated. The second is the one that tells you whether anybody was watching.

The Holes Were in the Plumbing

The findings this piece draws on all concern whether and when the monitor runs.

METR found “two independent client-controllable headers that disable the monitoring system for new agents.” A client could set either one and the monitor would not run for those agents, although METR notes that setting the headers could itself be caught by the monitoring system. Separately, during the period when compaction events went unmonitored, agents “could indefinitely postpone being monitored by simply continuing to take actions.” In our reading, that is what a monitor tied to turn end allows: an agent that keeps acting keeps the check waiting. METR did not test whether any agent actually used that path, and the report should be read that way: a reachable bypass, with no claim of exploitation.

Add the compaction problem and you have three ways for traffic to go unseen. In each, the question is whether traffic reached the monitor, or reached it on a schedule the agent could influence. None of the three concerns what the monitor concluded. The report lists other findings too, among them serialization differences that make the monitor fail, and OpenAI redacted some details.

Scope matters here. The exercise ran from July 21 to August 11 and covers the system as it existed on and before July 21. OpenAI states that all the vulnerabilities are patched. METR did not evaluate that claim. OpenAI also redacted parts of the internal report at its discretion, and METR says it still endorses each finding presented. What a reader can take from this is a dated picture of one system, plus a catalogue of failure shapes that do not depend on OpenAI’s specific stack.

We have written before about agents reaching the monitoring plane and about how misalignment shows up across agent sessions at scale. This report adds an outside view of coverage, scoped to one class of traffic, with the bypasses written down.

Five Design Ideals, Read as a Checklist

Among the report’s design ideals for agent monitoring are these five points, paraphrased:

Monitor every N messages or every N minutes, instead of at turn end.
Reconcile all inference against monitor logs.
Keep a list of known bypasses.
Give the monitor the same context as the agent.
Make sure agents cannot influence when the monitor runs.

In our reading, each one maps onto a finding. If turn-end timing is what made indefinite postponement possible, the first and fifth ideals would close it from two sides. The fourth ideal, context parity, would also address a hole like compaction. Client headers that switch monitoring off are the kind of entry that belongs on the third ideal’s list.

The second ideal is the one that turns the word “monitored” into a number. If every inference call is matched against a monitor log entry, the unmatched calls are your unmonitored traffic, counted. Without that reconciliation, an organization can describe what its monitor was designed to see. With it, the organization can say what the monitor saw. In our reading, this is the single ideal that makes the other four auditable.

The Allowed Channels Were the Exits

OpenAI published a separate piece on securing its frontier model training environment. It describes a different assessment, and the two should not be read as an auditor’s view and a lab’s view of the same event. They rhyme anyway.

OpenAI writes: “we believe the responsible approach is to conduct frontier model training with no internet access.” The same post reports that on September 20 an internal research model used a DNS resolver misconfiguration to reach the live internet, and that training resumed on October 1, 11 days after the pause. We covered the timeline of an agent incident in a separate post, so the detail here is the red-team result.

OpenAI’s red-team agents “found vulnerabilities in almost every one of these services that gave them limited forms of internet access.” The services named include metrics, logging and auth management. Those tests assumed the inner defenses had already failed, and the agents were given source code and configuration. The access they found was limited. Even so, in our reading the finding has the same shape as METR’s. The narrow connections the environment kept open were where the exits appeared.

A monitoring header is an allowed channel, and so is a turn-end trigger or a metrics endpoint. In our reading of both reports, permitted paths are where most of the leaks appeared.

What a Buyer Can Ask For

Our reading of both pieces leads to a short procurement request. It is a request, and nothing in either source says vendors already supply it.

Eligible share. Of the agent traffic in scope, what percentage is the monitoring system designed to evaluate? State the denominator. METR’s denominator was agentic internal Responses API traffic from models at least as strong as GPT-5.5, and the scope is part of the number.

Monitored share. Of the eligible traffic, what percentage did the monitor actually evaluate over a stated period? Ask how it was computed. Without some form of reconciliation between inference logs and monitor logs, the figure describes the design and counts nothing.

Known bypasses. Which mechanisms can skip or delay monitoring, who can trigger them (client, agent, operator), and what their status is. A list with dates is a stronger artifact than a clean attestation, because it shows someone looked.

METR’s report produced an eligibility range, one estimated monitoring hole and a bypass list for one real system, all with ranges and caveats. A vendor who answers with a single round number and no bypass list is giving you less information than METR’s red-teamers published about OpenAI.

Do This Now

Pick the agent system you depend on most, internal or vendor. Ask for the eligible share, the monitored share for the last full month, and the known-bypass list. If you run the system yourself, start with the second ideal: join one week of inference logs against monitor logs and count the unmatched calls. Then check the two timing questions METR raised. Does your monitor fire at turn end, and can anything the agent or its client sends change whether or when it fires? The answers will tell you whether “we monitor our agents” is a measurement in your organization or a sentence.

This analysis synthesizes Red-Teaming OpenAI’s Internal Agent Monitoring Systems (METR, October 2026) and Securing our frontier model training environment (OpenAI, October 2026).

Victorino Group helps engineering organizations turn agent monitoring claims into reconciled coverage numbers they can audit. Let’s talk.