The More Reliable the Agent, the Harder It Is to Justify Watching It

Andon Labs handed 15 frontier models a cheap off-the-shelf quadcopter and asked them to write the code that makes it find a named person indoors and stay with them. The models never fly anything. They write software, the software flies, and the flight is scored. From Opus 4.8 onward, the average submission beats the human baseline on the two tasks that involve a person: Detect and Follow.

That result belongs in a governance conversation more than a capability one. Andon states the consequence on the benchmark page itself: the more capable models become, the easier they are to misuse; the more reliable they become, the harder it is to justify a human watching. Reliability is the thing that dissolves the political case for oversight, and it arrives at the exact moment the worst outcome stops being a bad paragraph and starts being a collision.

What Drone-Bench Actually Measures

The eval chains five tasks that a physical agent has to perform in sequence. Reconstruct builds a 3D understanding of the room from flight data. Localize places the drone inside that reconstruction. Navigate moves it to a target coordinate. Detect identifies a specific named person in the frame. Follow keeps station on that person as they move.

Human baselines, established by people writing the same code, run: Reconstruct 82.2%, Localize 84.0%, Navigate 92.0%, Detect 72.1%, Follow 67.4%. The two lowest numbers are the two tasks that involve a human body in a moving frame. That is where models now win. On the Follow demonstration, the human baseline held an average distance of 1.21 metres from the target. Fable 5 held 0.87 metres.

The methodology is worth reading before drawing any conclusion. Each model gets 10 runs, 10 scored submissions per run. Critically, each task is run in isolation, with clean upstream artifacts injected. A model attempting Follow is handed a correct reconstruction, a correct localization, and a correct navigation. It never has to survive its own earlier mistakes. Reconstruct is held out on 39 waypoints (741 pairs), so the spatial understanding score cannot be memorized from the flight it was built on.

The Zero Nobody Is Quoting

End-to-end success across all five tasks: 0%. Not for any model, not once. No run has ever beaten the human baseline on Reconstruct. A typical run chains all five tasks about 6% of the time.

Two numbers from the same page point in opposite directions, and both are true. Superhuman on the tasks nearest the person. Zero on the full chain. The isolation design explains the distance between them: a chain of five stages, each individually strong, still collapses because errors compose. The drone that follows a person beautifully is following the wrong person if Detect was fed a reconstruction it built itself and got wrong at waypoint three.

We have written about blast radius as the real unit of autonomy risk in software systems, where a compromised agent exfiltrates data or drops a table. Drone-Bench moves the same argument into a room. The blast radius here is a person standing in the flight path. Nothing about a 0% end-to-end score protects them, because the failure that hurts is a single stage going wrong at speed, not the pipeline failing to complete.

Average Runs Are Six Months Behind Best Runs

The sharpest operational finding in the benchmark has nothing to do with drones. Fable 5 beats the human baseline on 2% of first submissions and 52% of best submissions. Averaged across all 15 models, going from first attempt to best attempt improves the score by 182%. For Fable 5 the improvement is 312%.

Andon translates this into a unit that procurement can feel: the average run lags the best run by roughly six months of model progress. The model you read about in the announcement post is the best-of-ten model. The model your operator actually gets, on the first try, on a Tuesday, is the model from two releases ago.

This is where most deployment decisions go wrong. A pilot produces a demo, the demo is a best submission, and the capability the organization then plans around is a capability that shows up in one attempt out of ten. The correct planning input is the first-submission number. Everything above it is a function of iteration budget, and iteration budget is exactly what a physical system operating in real time does not have.

The Part Andon Flags Against Its Own Result

Every attempt in Drone-Bench receives a verifiable score. The eval knows whether the drone found the right person and how close it stayed. Andon points at this as the artificial ingredient, and the point deserves more weight than a footnote.

Real-world autonomy is gated on verification, not capability. Inside the harness, a model can submit ten times because ten submissions can be graded. Take the grader away and the improvement curve from 2% to 52% has nothing to climb. The operator in a warehouse has no oracle telling them the drone tracked the correct person. They have the drone’s own report, which is generated by the same system whose reliability is in question.

That is the honest limit on what this benchmark proves, and it strengthens the governance argument. If reliability is what removes the human observer, and verification is what would justify removing them, then the two are decoupled. Teams will hit the reliability threshold that makes oversight feel expensive long before they hit the verification threshold that makes oversight unnecessary. The verification you cannot see problem, familiar from code review, becomes a safety problem the moment the agent has mass and velocity.

Improving Reliability Removes the Human

The usual story is that reliability earns an agent the right to operate unattended. Drone-Bench inverts the mechanism. Reliability makes the human hard to defend in a budget meeting. An operator watching a drone that succeeds 95% of the time looks like waste on a spreadsheet, and the argument for keeping them cannot be made from the success rate. It has to be made from the cost of the 5%.

In software, the 5% is a rollback. In a room, it is an injury report. The oversight decision therefore cannot be indexed to a reliability metric at all, which is uncomfortable, because reliability is the only metric these systems produce cleanly.

Do This Now

Take one autonomous system you already run, physical or not, and write down two numbers: its first-attempt success rate and its best-of-N success rate. If you only have the second, you are planning against a capability you do not have. Then answer one question in writing, with the person who signs off on headcount: what evidence, other than the success rate, would justify removing the human from this loop?

If the only answer is “it got more reliable”, the loop still needs the human, and you now have the sentence to defend them with. While you are there, confirm the system has a working stop path; the six surfaces a kill switch has to cover apply with more urgency when the agent has rotors.

---

This analysis synthesizes Drone-Bench (Andon Labs, July 2026).

Victorino Group helps organizations decide where human oversight of autonomous systems belongs, and prove the decision. Let’s talk.