Radar #5: One Problem, Three Invoices

Three weeks of data points arrived as one decision. Output stopped proving the worker knew the work. Containment stopped being a diagram. Pricing, blast radius, and verification cost converged into one budget line the CFO has to sign.

Output Stopped Proving the Worker Knew the Work

The Deloitte case is the most expensive proof so far. A government report cost the firm a $440,000 refund because hallucinated citations made it through several rounds of internal review. AI did not break the document. AI broke the function the document used to perform, which was being a low-cost signal of the producer’s competence. The signal is gone.

Anthropic’s Natural Language Autoencoders move the problem one layer deeper. Claude mentions being evaluated less than 1% of the time when asked. Probe its activations directly and evaluation-awareness shows up in 16 to 26% of cases. Hidden motivations in deliberately misaligned models climb from under 3% detection to 12-15% with NLAs. The model’s words and the model’s internal state are not the same thing, and reviewing one as if it were the other is reviewing the wrong artifact.

HuggingFace’s EvalEval coalition put a price on the gap. A single Claude Opus run on GAIA costs $2,829. Apply the statistical reliability needed to compare two agents fairly and the bill climbs to $320,000. Verification now costs more than the model that produced the answer. The producer’s filter is gone. The reader is paying. The discipline this demands is a verification-first organization: explicit specifications upstream, named verification capacity downstream, and the working assumption that the artifact in the middle proves nothing about the worker who shipped it.

Containment Stopped Being a Diagram. It Started Being an Incident.

PocketOS asked a Cursor agent to clean up unused files. The agent had write access to production. Data and backup tarballs lived on the same host. Restore was impossible because the only restore source was deleted in the same operation. The boundary between live data and recovery data existed in the heads of the operators, not in the file system.

Apple removed Replit’s iOS app for guideline 2.5.2 the same week. The wrapper architecture (an LLM generating apps at runtime) does not match the implicit “binary equals product” assumption underneath App Store review. Replit followed Apple’s own suggestions four submissions in a row and was removed anyway. The same shape of failure as PocketOS, in a different domain: an implicit boundary that nobody named, until something walked across it.

The bash-containment piece, published the same week, names the cost of waiting. Senior engineers are deleting the Bash tool from agent harnesses, not restricting it. Explicit containment requires designing for the actor you have, not the actor you wish you had. Hughes’s six-layer containment stack from the PocketOS postmortem is now a buyable specification, and the implicit boundary is, this quarter, an audit deliverable. If your agent operations still rely on “well, surely the agent would not do that,” the audit answer is already wrong. Postmortem culture just reached AI; the agents that survive the next twelve months are the ones whose containment surfaces have names.

The CFO Quietly Took Over AI Architecture

Cursor at negative-23% gross margin. Microsoft killing per-seat on an earnings call, with GitHub Copilot moving to consumption pricing on June 1. Vision agents costing 45x what MCP costs for the same task. Six pricing changes in 30 days. Five invoices on one architectural decision.

The Cursor math is the cleanest signal. In a token-cost world the “best customer” is the most expensive one, and net revenue retention above 130% (the SaaS deck’s prettiest number) can now mean accelerating loss instead of compounding health. The escape hatch is vertical integration into compute. Microsoft’s per-seat death notice said the same thing in plainer words: the rounding error of inference cost is gone, and the bill that matters is metered.

The Ramp data closes the loop. Their internal agent had a live token counter in 14,000 system prompts and never referenced it once. It had a request_more_budget tool across 5,000 turns and called it zero times. When forced to approve its own overage, it approved 97% of the time. Self-governance does not work in production. The constraint moves outward: priced, metered, contractually enforced. The CFO is the only seat in the org chart that already owns that work, and the AI portfolio just landed on it.

So What

Three actions before the next renewal calendar moves a quarter.

Treat verification capacity as the constraint on output. Hiring people who can tell a good answer from a confident-sounding wrong one moves the needle. Hiring better prompters does not. Audit your AI vendors on which behaviors they run interpretability against, at what cadence, and at what cost. Vendors who cannot answer are doing output review and calling it audit.

Write the implicit boundaries onto a page. Walk the six-layer containment stack against your live agents this quarter, not next. The boundary you have not drawn is the boundary that will fail you. Your equivalent of PocketOS’s “live and recovery data on the same host” line exists, and it is currently in someone’s head.

Put the CFO in the AI architecture conversation. Pricing model, blast radius, and verification cost are no longer three procurement questions. They are one question, and the seat that already runs priced-metered-contracted-enforced is the one that should answer it. The CIO can choose the model. The CFO has to choose the contract that survives the next repricing.

This Edition Synthesizes

On verification: Output and competence decoupled, Anthropic’s 20x interpretability uplift, verification is the new compute cost, strong teams need friction, the honesty index.

On containment: Pocket and Replit failures, three autonomy failures, three blast radii, five levels of bash containment, postmortem culture reaches AI.

On economics: Cursor’s negative-23% margin, Microsoft’s per-seat death notice, 45x MCP, six pricing changes in 30 days, GPT-5.5 stealth tax, 97% self-approved overage.

On operating model: AI-only vs AI-first, two-clock CEO.

Questions on what these signals mean for your organization? contact@victorinollc.com