A Carnegie Mellon study, presented by Tariq Shaukat at the AI Engineer conference in July 2026, tracked what happens to teams after they adopt AI coding agents. Month one looks like the vendor demos promised: a 3-5x velocity boost, real and measurable. By month three, the boost is gone. Velocity is back at baseline, and the study identifies why: security issues, maintainability issues, reliability issues, and complexity all climbed while the code was being produced faster. The gain was genuine. It just had a half-life.
One caveat up front. Shaukat is the CEO of Sonar, a company that sells verification tooling, and the study has no public URL; it exists in this analysis as he presented it. The shape of the curve still deserves attention, because it matches what independent data predicts and what many engineering leaders are quietly observing: the initial acceleration is real, and so is the reversion.
We have written about the downstream effects of AI-generated code volume before: how faster code writing jams the rest of the delivery pipeline, how adoption rates race ahead of hours actually saved, and how developers feel more productive while their working experience degrades. The Carnegie Mellon curve adds the dimension those pieces did not have: time. The gain exists at month one and is gone by month three, and the mechanism of its disappearance is knowable.
Debt Compounds at Machine Speed
The mechanism is accumulation. An agent that writes code five times faster than a human also introduces security findings, tangled abstractions, and untested branches five times faster. Nothing about the generation process filters them out, because the agent optimizes for the task in front of it, and the human who used to catch these problems in review is now approving five times the volume with the same number of eyes.
For the first few weeks, none of this is visible. The codebase absorbs the defects the way any codebase does, silently. Then the interest payments start. Bug triage takes longer because the surface area grew. New features touch code nobody fully reviewed. Security backlog tickets multiply. Every hour that used to go into new work gets redirected into firefighting, and the team’s measured velocity slides back toward where it started.
Speed without verification behaves like a loan. The 3-5x boost is the principal, spent immediately and enjoyably. The decay back to baseline is the repayment schedule, and the interest rate is set by how much unverified code entered the system during the fast months. Teams do not lose the gain because the agents stopped working. They lose it because they spent three months borrowing against their own future review capacity.
The 50% Engine Under the Curve
METR’s task-horizon research explains why the debt accumulates so reliably. In “Measuring AI Ability to Complete Long Tasks” (March 2025), METR measured how long a task can be before frontier models fail it. The headline number: models now complete tasks that take humans roughly 16-18 hours. The fine print: at a 50% success rate. Require 80% accuracy instead, and the task horizon collapses to roughly 3.5 hours, a number Shaukat’s updated version of the curve reiterated.
Read those two numbers together and the three-month decay stops being surprising. A team that hands its agents ambitious, day-long tasks is operating in the region where half the outputs contain failure. Those failures rarely announce themselves; they compile, they pass the happy path, they get merged. Each one is a deposit into the debt account.
Shaukat quoted a CTO who put the enterprise view plainly: an employee who gave him information that was 80% accurate would end up on a performance review. Enterprises do not run on 80%, and they certainly do not run on 50%. The distance between what models can attempt and what businesses can accept is exactly the space verification has to fill.
Per Sonar’s own benchmarking across more than 4,000 problems, state-of-the-art models score high on functional correctness while still generating bugs, security issues, and wildly variable complexity. Treat that as a vendor’s self-reported number with the obvious incentive attached. It is nonetheless consistent with METR’s independent measurement: capability and reliability are different axes, and the second one is where the decay lives.
What the Surviving Teams Buy With the Gain
Some teams keep the velocity. The pattern they share, in Shaukat’s framing, is a loop of guide, verify, and solve, with verification wired into the agentic workflow rather than bolted on after it.
Guiding means the agent starts constrained: project context, coding standards, architectural boundaries, and explicit definitions of done, supplied before generation begins. Sonar reports that customers using context-plus-constraints guidance cut token consumption by over 30%, which tracks with intuition: an agent that knows the rules wastes less effort exploring paths that will be rejected.
Verifying means two layers. Deterministic checks first: static analysis, security scanning, type systems, policy gates. These are cheap, fast, and incapable of being charmed by plausible-looking code. Agentic review second, for the properties deterministic tools cannot judge: design fit, requirement coverage, whether the change does what the ticket meant. The layering matters because each catches what the other misses.
Solving means the findings feed back into the loop automatically, so the agent fixes its own output before a human ever sees it. The human reviews an artifact that already survived the gauntlet, which is the only way review scales to machine-speed volume.
The outcome numbers here are Sonar’s own, reported about its customers, so hold them loosely: AI-derived production outages 44% less frequent among teams using multi-layered verification, and one large bank measuring a 92% reduction in issues after adopting the full loop. Discount the magnitudes as much as you like. The direction is the argument, and the direction agrees with the CMU curve read in reverse: if unverified debt is what eats the gain, verification is what preserves it.
Clean Code Became an Input Cost
Shaukat’s most useful observation may be the least promoted one: agents work measurably better on clean codebases. The same agentic task on a cleaned-up codebase consumes materially fewer tokens and less reasoning than on a typical one. Clear structure, low duplication, and consistent patterns are things the model can exploit; tangled legacy confuses it into longer, more expensive, more error-prone runs.
This inverts a long-standing economic judgment. Refactoring and maintenance used to be a cost center justified by developer morale and long-term risk arguments that CFOs tolerated at best. In an agentic world, codebase health is an input cost with a measurable exchange rate: cleaner code means cheaper tokens, shorter runs, higher success rates, and less debt per generated line. Quality is no longer only what you hope comes out of the pipeline. It is what determines the pipeline’s efficiency going in.
The compounding works in both directions. A team that spends part of its velocity gain on maintenance makes every future agent run cheaper and safer. A team that spends none of it makes every future run worse, which is the CMU decay curve described from the inside.
Do This Now
Chart your own half-life. Pull your delivery metrics from the month you adopted coding agents and plot velocity against your security and maintainability backlog, month by month. If the backlog slope turned upward while velocity flattened, you are on the curve, and you now know your position on it.
Then reinvest deliberately. Take a fixed fraction of the velocity gain, a quarter is a defensible start, and spend it on the loop: context and constraints supplied to agents before generation, deterministic checks plus agentic review inside the workflow, and scheduled maintenance on the code the fast months produced. The teams still holding a 3x gain at month twelve will be the ones that treated verification as part of the production line. The gain is real. Whether it decays is a design decision.
This analysis synthesizes In the Land of AI Agents, the Verifiers Are King (AI Engineer, July 2026) and Measuring AI Ability to Complete Long Tasks (METR, March 2025).
Victorino Group helps engineering organizations build verification into their agentic workflows so the velocity gain survives past month three. Let’s talk.
All articles on The Thinking Wire are written with the assistance of Anthropic's Opus LLM. Each piece goes through multi-agent research to verify facts and surface contradictions, followed by human review and approval before publication. If you find any inaccurate information or wish to contact our editorial team, please reach out at editorial@victorinollc.com . About The Thinking Wire →
If this resonates, let's talk
We help companies implement AI without losing control.
Schedule a Conversation