Three Queues in the Harness, Eighteen in Production. The Agent Was Right.

Checkly’s local environment routed retries to three queues, selected by check type alone. Production routes them differently: there, routing also depends on factors like priority and hosting type, for a total of 18 possible queues per region. The agent wrote code that was correct against the three. The component it was rebuilding processes approximately 92,000,000 messages every day, around 40,000,000 of them check results and the rest WebSocket publishes.

Every gate that code passed was built on the same premise the code was built on. The harness defined what correct meant, the agent satisfied that definition, and the definition described a system that did not exist.

Checkly published that defect inside a rewrite that otherwise reads as a win. A Node.js service became a deployable Go service of about 13,000 lines of application code. It shipped with zero incidents, a 70% reduction in running pods, and a lighter database load: 60% fewer total average active sessions on the database and a roughly 15% reduction in database CPU, and freed around 15 vCPU and 45GB of memory. Token usage stayed within the daily limits of a $200 subscription.

Three teams arriving at the same architecture

Checkly states the design principle plainly. “If an agent is going to write the code, something other than a human reviewer has to define what correct means,” writes Edvinas Janusevicius, backend engineer at Checkly. The definition of correct moves out of a person’s judgment and into an artifact that executes.

Shopify built the same move into a checkpoint. Each checkpoint in their Helix workflow must “prove its behavior with tests, match the running app in a visual review, survive two adversarial code reviewers, and get a human’s nod.” Shopify reports the Shop app went “from a proof of concept to a fully rebuilt native app published in the app stores in just 12 weeks.” Shopify names the binding constraint directly: “It doesn’t matter how good the model is if it can’t test its work quickly.” Their 2020 decision against native was reversed, and the reasoning they give has the same shape. “Native still means building and maintaining software on two platforms, that cost has not disappeared. What changed is that agents can now do enough of the implementation, translation, testing, and review work that it’s no longer the deciding factor it was in 2020.” Separately, they describe their main app as “our biggest with 300+ screens.”

Meta’s Second Brain work, published in September 2026, lands on the review half of the same architecture. The team organized 200+ files into a strict taxonomy and cut tokens consumed per turn by around 80%, and after three development sprints spanning six weeks reported zero regressions across improvement cycles to the knowledge base. The review unit is deliberately small: “Every improvement is a text edit that a domain expert can review in 30 seconds.” And the reviewer is quarantined by construction. “A separate agent, running in a fresh context with no knowledge of the improvement rationale, receives only the proposed diffs … Because it shares no context with the proposing agents, it cannot inherit their blind spots.” This is a different system from the DrP debugging platform we covered in codifying institutional intelligence, and the two should not be read as one programme.

Three organizations, three products, one structural answer: build the definition of correct before the agent writes, and make review something other than a person reading a diff.

Context isolation has an upstream limit

Meta’s fresh-context reviewer solves a real and specific problem. A reviewer that hears the proposer’s rationale tends to adopt it. Strip the rationale, hand over only the diffs, and the verdict comes from the standing rules instead of the story. That is a genuine independence, and it is independence from the agent.

That independence stops at the harness. A reviewer reading only the diff has nothing to compare the diff against except the encoded description of the system. If that description says three queues, a perfect reviewer confirms three queues. Shopify’s two adversarial reviewers hit the same ceiling for the same reason: adversarial review interrogates whether the change does what the checkpoint claims, and the checkpoint is the thing that is wrong.

We have argued that the harness is the control surface worth governing and that the durable fix belongs in the loop rather than the output. Checkly’s defect adds the boundary condition to both. A harness governs only the part of production it models accurately. Where the model diverges, the harness stops being a control and becomes an amplifier, converting one human misunderstanding into compliant code at machine speed.

This is the inverse of the failure we described in agents gaming verification at scale. There, the agent finds the cheapest route past a gate and the gate is the victim. Here the agent did the work honestly, the gate fired correctly, and the verdict was still worthless. No amount of adversarial pressure on the output recovers it, because the pressure and the output share a premise. It also sits close to the pattern in silent drift in agent operations: the system keeps reporting green while the thing it measures has moved.

Written down, and verified against what

Checkly’s post-mortem rule is one sentence: “Every assumption about the surrounding infrastructure must be written down and verified, not inferred.”

Both halves do work. Written down first, because an assumption that exists only as local configuration is never read as an assumption. Three-queue routing was a fact of the local environment. It behaved like the truth because nothing in the loop ever asked it to identify itself as a claim.

Verified second, and the harder question is verified against what. The harness was built from somebody’s understanding of production. Checking the harness against that understanding confirms the copy against the original error. Verification that means anything has to reach the running system: the queue names the region actually has, the routing keys it actually evaluates, the environment variables production actually sets, the grants the local database never enforces.

The economics of getting this wrong changed with the throughput. A person building against a false premise produces a bounded amount of wrong code before someone notices. An agent works at the scale of the whole service, about 13,000 lines of application code, and a premise buried in the harness propagates across all of it before anyone reads a line. The defect is not more likely under agent authorship. It simply reaches further before it surfaces, and it arrives wearing a full set of green gates.

Do this now

Spend an hour with the harness that your agents write against, and produce a list. Not of what the code does, of what the harness asserts about production:

• Queue, topic and partition counts, and every key that selects between them
• Environment variables production sets that the local environment does not
• Database roles and grants the local instance leaves unenforced
• Feature flags, and their real production values
• Anything that varies by region, tenant or hosting tier

Then verify each line against the running system rather than against the document the harness was built from. Where verification is impossible today, write the assumption into the repository with a name attached to it, so the next agent inherits a claim instead of a fact.

The review target has moved. The question I kept asking was whether the agent’s output could be trusted. The question now is whether the harness that judges the output describes the system you actually run.

---

This analysis synthesizes Rewriting a Node.js Service in Go With AI Agents (Checkly, 2026), Native is now the future of mobile at Shopify (Shopify Engineering, September 2026), An Organizational Second Brain: Building an AI That Learns From Experts (Meta Engineering, September 2026).

Victorino Group helps engineering teams audit what their agent harnesses assume about production before those assumptions ship. Let’s talk.