Correctness Is the Easy Half: Two Computable Measures of the Other Half
Verbosity: 0.15 plus or minus 0.06 in established repositories, 0.33 plus or minus 0.10 in agent-generated code. Erosion: 0.31 plus or minus 0.17 against 0.68 plus or minus 0.20. Those four numbers come from SlopCodeBench, and the post that published the two formulas behind them summarizes the comparison in one line: “The agent’s code is on average roughly twice as verbose and eroded as human code.”
The formulas matter more than the numbers. Both are computable this afternoon with tooling you can install, and both produce a single ratio per repository, which is the shape a CI gate can consume.
The half nobody computes
Getting a model to generate code and letting hidden tests check it is straightforward. The author of Measuring code sloppiness states the problem with the other half plainly: checking the sloppiness of that code “often requires human intuition and time, and is an extremely difficult task in general.”
His framing of what correctness leaves on the table: “Just because the code is formally correct doesn’t mean that it is not introducing unnecessary abstractions, creating duplicates, or just making bad decisions overall.”
We reported the Sonar benchmark across 53 models and gave a qualitative taxonomy of how agents decay a codebase. Neither of those carried a formula a reader could run. These two do.
Formula one: Verbosity
Verbosity = |flagged_lines ∪ clone_lines| / LOCflagged_lines is the set of lines caught by AST-Grep rules. clone_lines is the set of lines a clone detector marks as duplicated. Use the union. A line that is both duplicated and flagged counts once, where a sum would double it. Divide by total lines of code.
The metric captures duplication and needless verbosity in a single ratio. Tooling is off the shelf: AST-Grep plus a clone detector. No model in the loop, no judgment call in the measurement, no per-review cost that scales with volume.
Formula two: Erosion
Erosion measures structural decay into large tangled functions. It starts by assigning each function a mass:
mass(f) = CC(f) * sqrt(SLOC(f))CC(f) is the cyclomatic complexity of the function. SLOC(f) is its source lines. The square root damps the length term so that complexity weighs more heavily than raw length. Read the effect on an invented pair: a 200-line function of straight-line assignments is a different animal from a 60-line function with fourteen branches.
Then:
Erosion = sum of mass(f) for all f where CC(f) > 10
-------------------------------------------
sum of mass(f) for all fThe result is the fraction of the codebase’s structural mass that sits inside functions above the complexity threshold. The threshold, CC > 10, is the copyable parameter. It is the one number in the pair that a team can defend or change with an argument.
Both formulas share a design choice worth noting: they normalize. A repository that triples in size does not automatically triple its score. That is what makes them usable as a trend line rather than a one-off audit.
What the benchmark values do and do not say
SlopCodeBench is a third party’s work, cited by the post that supplies the formulas. Credit runs in two directions: the benchmark to the paper, the two measures to the post.
| Metric | Established repos | Agent-generated |
|---|---|---|
| Verbosity | 0.15 plus or minus 0.06 | 0.33 plus or minus 0.10 |
| Erosion | 0.31 plus or minus 0.17 | 0.68 plus or minus 0.20 |
Carry the spreads every time you quote these. “0.33” alone is a different claim from “0.33 plus or minus 0.10,” and the erosion bands in particular are wide enough that a single repository sitting near a boundary tells you very little on its own. The roughly-two-times ratio is arithmetic on the means, and the source already states it; it is not additional evidence.
Treat these values as a reference point for calibrating your own baseline. They stop short of being a threshold. Nothing in the source recommends a value at which a build should go red, and inventing one would be the kind of precision the data does not support. Measure your own repositories first, over several months of history, and set the gate from what you see.
The slot these formulas fill
Rachel Laycock, CTO at Thoughtworks, published her case against reviewing all this code the same month. Her description of the bottleneck: “If an agent can produce ten times the code but every line eventually queues up waiting for a senior engineer to inspect it, we haven’t created a ten-times engineering organisation, we’ve created a big backlog and a new bottleneck.”
Her proposed replacement for human review is to “encode the important constraints as fitness functions.” She names the anti-pattern that hides in the alternative: “an AI agent pretending to be the human reviewer so we can preserve exactly the same process at higher speed. That’s automating the ceremony rather than questioning why the ceremony exists.”
We argued the review-by-exception position already and will not re-litigate it. Verbosity and Erosion are candidate fitness functions for the slot that argument leaves open. They sit alongside the rest of Laycock’s practice stack: pair programming, trunk-based development, automated testing, static analysis, security scanning.
Laycock also keeps a trigger list for what still reaches a human: a fundamental architectural change, something crossing a sensitive security boundary, a change with a huge blast radius, an unfamiliar part of a critical system, or “simply something where the team says, I’m not confident about this.” Neither formula touches any of those. They measure shape. Intent stays outside their reach.
She concedes the strongest counter-argument, from Brian Houck at DX: teams accumulate “cognitive and intent debt: software grows while the humans responsible for it understand less and less about why it works the way it does.” Laycock agrees the debt is real and disputes only that mandatory pull requests defend against it. A rising Erosion number is one of the few signals that makes part of that debt visible before the people who could explain it leave.
The Goodhart warning ships with the metric
The author cites Goodhart’s Law explicitly, and is direct that final judgment still needs a human. That caveat is the most credible thing about the pair.
The obvious way to game Verbosity is to remove duplicated lines and keep the same bad decisions, rearranged. The obvious way to game Erosion is to split complexity across more functions and relocate the tangle into the call graph. The metrics are instruments for noticing, and the noticing is what earns the human’s attention. They do not replace it, and the source does not claim they do. Neither source documents anyone running these measures in production.
We have made the case for imposing values rather than disciplines on agents. A metric is a discipline. The value it serves is that a codebase should stay comprehensible to the people who own it, and the metric is only useful while it tracks that value.
Do this now
Run both formulas over your last six months of history, one data point per week, on a repository where agents write a meaningful share of the code. That is an afternoon of work: AST-Grep, a clone detector, a complexity tool, and a script that walks the git log.
You are looking for a slope. A single week’s score tells you nothing on its own. A flat line means your existing practices are holding. A rising Erosion line with a flat Verbosity line means complexity is concentrating while duplication stays controlled, which points at design review rather than cleanup. Both rising means the review process has stopped catching structure.
Set your gate from the slope you measure, not from the benchmark values. The benchmark tells you what a healthy repository and an agent-heavy one looked like in somebody else’s sample. Your own baseline is the only number your build should fail on.
This analysis synthesizes Measuring code sloppiness (Sebastian, earendil.com, September 2026), SlopCodeBench (arXiv 2603.24755, 2026), and Maybe we shouldn’t be reviewing all this code (Rachel Laycock, Thoughtworks, September 2026).
Victorino Group helps engineering organizations build fitness functions and quality gates for agent-written code. Let’s talk.
All articles on The Thinking Wire are written with the assistance of Anthropic's Opus LLM. Each piece goes through multi-agent research to verify facts and surface contradictions, followed by human review and approval before publication. If you find any inaccurate information or wish to contact our editorial team, please reach out at editorial@victorinollc.com . About The Thinking Wire →
If this resonates, let's talk
We help companies implement AI without losing control.
Schedule a Conversation