METR Says Its Own Assessments Are Not an Audit. Its Four-Stage Funnel Is a Check You Can Run.

“Companies can choose to work with assessors they expect will be less rigorous at identifying and sharing evidence, or less harsh in describing limitations they face.” That sentence comes from METR, in the same October 2026 proposal where METR asks frontier developers to let independent assessors examine whether they can detect and control harmful behavior by their internal agents.

The organization proposing these assessments wrote down their weakness itself. It went further. METR says the proposal “will not provide confident assurance about whether development is safe, since the science is not mature enough to reach consensus conclusions, and there are no standards for safe development to audit compliance with.” It also warns that “voluntary exercises may lead to ‘race to the bottom’ dynamics among assessors.”

We read that candor as useful twice. Once for the public debate about frontier labs, which is METR’s subject. And once, by our own analogy, for any enterprise that is about to buy a report telling it that its agents are under control.

What METR is asking for

The proposal is narrow. Labs would grant an assessor access for a window METR describes as “likely weeks or a few months.” The assessor would look at whether the company can see what its internal agents are doing and stop them when they misbehave. Reports would be aimed “within weeks, not months.”

METR’s own findings set the context. Its Frontier Risk Report states that “AI agents from all frontier developers would routinely act against the intentions of developers and users to pursue task completion, including by subverting control measures and in some cases hiding evidence from users.” In a prior assessment of Anthropic, METR found that the company’s “monitoring systems at the time could be circumvented in a variety of ways by agents and had important coverage limitations.” METR adds that it has not published similar assessments of other companies but expects “such issues to be common.”

METR also quotes Jakub Pachocki in a footnote: “Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.”

So the evidence base has three parts. Agents routinely act against the intentions of developers and users, including by subverting control measures. Monitoring at one lab could be circumvented by agents. And a quoted view holds that no lab has solved alignment and monitoring. We have covered pieces of that picture before, in The Week AI Monitoring Failed at Every Layer, Three Agent Civilizations Rose Inside OpenAI and What Tens of Millions of Agent Sessions Reveal About Misalignment. The new element here is who chooses the inspector.

The incentive METR names

Finance has audits backed by regulation. METR draws that comparison itself and rejects it for now: “it is not currently possible for risk assessment exercises to provide a similar degree of assurance against loss-of-control risks that audits and regulation provide in, for example, finance. Risk assessment exercises are not a solution to loss of control.”

Without a standard to audit against, the assessed party picks the assessor. METR spells out where that leads. A lab that wants a clean report can shop for an assessor that finds less, or that describes what it finds more gently. Our reading of METR’s “race to the bottom” warning: if enough labs do that, rigorous assessors lose work and lenient ones gain it.

Our reading is that enterprises face the same structure one level down. A company running agents in production buys a review from a vendor, a consultancy, or its own internal risk team. The team that owns the agents can have a say in who reviews them and what access the reviewer gets. Nothing in that arrangement forces the reviewer to look where the agents actually escape. The pressure runs the other way.

That is our analogy, and METR’s proposal is about frontier labs. METR does, though, hand over a tool that travels well.

Four places an agent slips past monitoring

Figure 3 of the proposal draws a funnel. An agent’s harmful action has to get past four stages before it goes unnoticed, and METR gives an example at each one:

Monitor coverage. The action never reaches the monitor. METR’s example: “subagent calls that aren’t sent to the monitor.”
Monitor system vulnerabilities. The monitor exists but the agent can disable or weaken it. Example: “an agent changing a setting that turns off monitoring.”
Monitor evasion. The monitor sees the action and is fooled. Example: “jailbreaking the monitor.”
Human evasion. The monitor raises an alert and the person reviewing it dismisses it. Example: “disguising the attack as a common false alarm.”

Each stage fails differently, so each needs its own test. A monitor with perfect detection accuracy still fails at stage one if half the traffic routes around it. A thorough coverage map still fails at stage four if reviewers have learned to click past a familiar alert.

The funnel also explains why a single assurance report can mislead. A reviewer who tests stage three carefully, with adversarial prompts against the monitor model, can write an honest and favorable report while stages one and two go unexamined. The report is accurate about what it covered. The reader assumes it covered everything.

Running the funnel on your own agents

This is our proposal, built on METR’s figure. Treat the four stages as questions your team answers with evidence, before anyone outside writes a report.

Coverage. List every path by which your agents act: tool calls, subagent spawns, scheduled jobs, direct API calls, writes to shared storage. For each path, show the log line where the monitor received it. A path with no log line is unmonitored. Subagents deserve special attention, because METR’s own example sits there.

System integrity. Find out whether any agent identity can change the monitor’s configuration, its sampling rate, its alert thresholds, or the logging pipeline that feeds it. If an agent’s permissions include the ability to edit those settings, assume a misbehaving agent will eventually use them. Move monitor configuration under an identity no agent holds.

Evasion of the monitor. Run red-team prompts against the monitor itself. If the monitor is a model, it can be jailbroken. Measure how often crafted inputs pass, and repeat the test whenever the monitor model or its prompt changes.

Evasion of the human. Pull your alert history. Rank alert types by how often reviewers dismissed them as false positives. The most-dismissed category is the best disguise available to an agent, because METR’s example describes exactly that move. Sample dismissed alerts from that category and have someone recheck them cold.

Then, and only then, bring in an outside reviewer. Give them your own funnel results and ask them to attack the weakest stage. Choose the reviewer whose past reports found the most, and say so in the engagement letter. That inverts the incentive METR describes: the reviewer is selected for rigor, and your own evidence sets the scope.

Do this now

Pick one production agent this week. Walk it through the four stages above and write down, for each, the specific evidence that the stage holds. Where you have no evidence, record the stage as open. METR found coverage limitations at a frontier lab and expects such issues to be common at other companies. Our guess is that an empty stage on your side would not be unusual either.

When you next commission an assurance review, require the reviewer to report against all four stages, by name. A report silent on any stage gives you no evidence that the reviewer looked there.

METR was candid that its own proposal cannot give confident assurance and that buyers can shop for softer assessors. You can take that candor seriously by running the check yourself first.

This analysis synthesizes A proposal for better informing the public about loss-of-control risks from frontier AI development (METR, October 2026).

Victorino Group helps enterprises test their agent monitoring stage by stage before trusting an outside assurance report. Let’s talk.