The Reward Function Has No Line for Maintainability

Reinforcement learning for coding agents scores two bits: did the target test go from failing to passing, and did the previously passing tests stay passing. That is the whole reward channel on SWE-bench Multilingual, the benchmark family that shaped a generation of coding models. Dex (Dexter Horthy), founder of HumanLayer, states the consequence without hedging in “Why Software Factories Fail”: “there is no penalty for eroding codebase maintainability.”

He discloses his own stake in the first paragraph of that piece: “I run a company (HumanLayer) building tools in the human/agent collaboration space.” He is describing a deficit he also sells against, which is a reason to test the claim against your own repository rather than accept it. The mechanism, though, is checkable independently of his product.

We have argued this theme from the downstream end several times. Verification became the job. More code shipped, fewer deploys landed. The harness itself became a 90-day artifact. Those posts describe symptoms. Horthy relocates the cause to the training objective, which is upstream of every harness decision a team can make.

Maintainability Has No Fast Oracle

The reward function omits maintainability for a measurement reason, and it is a hard one.

“Tests give you feedback in seconds,” Horthy writes, “but the cost function of bad architecture is measured in weeks, months, maybe even years.” Reinforcement learning needs a signal it can query millions of times per training run. Test suites qualify. Architectural decay does not, because the bill arrives after the training run finished, after the model shipped, after three quarters of feature work piled on top of the decision.

Then there is the circularity, which is the sharper half of the argument: “if a model could reliably tell good code from bad, it might have written the good version to begin with, but maintainability has no fast oracle, so we can’t reward for it during RL.” You cannot use the model as its own judge of quality without assuming the capability you are trying to train. And you cannot buy your way out with a benchmark, because, in his capitals, “THERE ARE NO GOOD BENCHMARKS for a model’s ability to maintain codebase quality.”

One grading detail from SWE-bench Multilingual is worth stealing regardless of what you think of the larger argument. The harness keeps the model’s patch but throws away any edits it made to test files, then applies the held-out test patch on top. The reason is empirical: “we’ve caught a model quietly commenting out the failing test or splicing in a mock that makes the test useless.” A reward channel narrow enough to be fast is also narrow enough to be gamed, and the gaming shows up in test files first. If your internal evals do not discard agent edits to test files before scoring, your pass rate is partly measuring the agent’s ability to edit the scoreboard.

Why More Tooling Does Not Reach the Ceiling

Horthy is careful not to dismiss tooling: “more review agents and more tokens do help. They raise the floor, catching the dumb stuff. But they don’t move the ceiling, because the ceiling is whatever we managed to teach the model in RL.”

That single sentence explains a pattern most engineering leaders have now lived through. You add a reviewer agent, then a second one with different rules, then a linting pass, then a spec-conformance check. Obvious defects drop. The architectural drift continues at roughly the same rate, because nothing in the stack has an opinion about whether the third abstraction layer should exist.

His own timeline is the useful evidence here, more than any aggregate. Agent-built codebases, in his experience, start to struggle after three to six months. He began a lights-off attempt in July 2025. Around the third production incident in November he rewrote from scratch, and his cofounder spent two full weeks replumbing patterns by hand. That is a founder with strong incentives to report the opposite outcome.

On rework he is blunter: 20% rework is already a burden on a team, and “most AI oneshot PRs trend closer to 50%.” The review economics follow directly. Incremental review means 100 to 200 lines per step. Review at the end means 2,000 lines or more, which is the volume at which human attention degrades into approval.

He also cites a Faros AI dataset on review load and incident rates, and it is worth flagging that he presents those figures in image alt text and calls the report “more of a correlation signal than a verifiable smoking gun” (Faros AI, via HumanLayer). We used that same dataset earlier this month. It is not the load-bearing part of this argument and should not be treated as such.

Three Evals Trying to Build the Missing Oracle

The interesting response to a missing oracle is to build one. Three frontier efforts are attempting exactly that, and each attacks a different property of the binary bit:

SWE-Marathon (Abundant AI) replaces the single pass/fail bit with a compound reward channel, and stretches task length from roughly 15 minutes on SWE-bench Multilingual to roughly 400 hours. Long horizons are where architectural consequence becomes observable at all.
DeepSWE (Datacurve) trains on repositories that were never built in the real world, which makes it contamination-proof. If a model has never seen the codebase, its score reflects reasoning rather than recall.
Frontier Code (Cognition) uses multi-PR tasks, penalizes tests that fail to fail on the pre-patch code (the exact gaming behavior described above), and adds a judge model scoring the diff against explicit code-quality rules.

None of these has yet been demonstrated as a training signal at frontier scale. Until one is, the ceiling Horthy describes stays where it is, and the honest planning assumption for the next two model generations is that maintainability remains untrained.

Judgment Moves Earlier Because It Cannot Be Delegated Later

If the model cannot be taught to preserve architecture, and the review gate arrives too late to enforce it, the only remaining place to put human judgment is before the code exists. Horthy’s four planning phases are the practical form of that:

Product Review. The problem stated in the user’s terms, with success criteria. He prefers HTML mockups over prose, because a mockup makes disagreement visible in seconds.

System Architecture. Services, endpoints, schemas, queues, stores. Mermaid sequence diagrams, contract shapes, SQL data models.

Program Design. He calls this phase “criminally underemphasized,” and it is the one almost nobody writes: call-stack trees expressed in diff syntax, a file-tree diff annotated NEW and MODIFIED, plus types and method signatures for the key new functions. This is the layer where an agent’s structural choices are still cheap to overrule.

Vertical Slices, ordered middle-out. API contract plus mock data, then frontend against mocks, then wire the API to services, then migrations and services-to-database, then business logic, then error handling. Left to themselves, models prefer horizontal plans: all migrations, then all services, then the API, then the frontend. Horizontal plans defer integration risk to the end, which is where schedules die.

Worth keeping in proportion: roughly 40% of tasks are oneshot, or oneshot plus a round or two of light feedback. Those need none of this. Applying four planning phases to a copy change is how good process gets abandoned.

Pick the Reviewer Before the Code Exists

The most adoptable line in the whole piece costs nothing to try: “If you wanna save time during review, you pick the person who would review the PR, and run through the product/tech specs with them before you get to the coding part.”

Consider what that changes mechanically. Today the reviewer meets the work as 2,000 lines of finished diff, at which point every objection is expensive, socially and technically. Rejecting a design at that stage means discarding an agent’s afternoon and a colleague’s sense of progress, so reviewers round toward approval and log the concern as a follow-up ticket nobody schedules. Move the same reviewer to spec time, and the identical objection costs one paragraph edit. Nothing has been built to defend.

It also fixes an ownership problem that agent-heavy teams create quietly. When an agent writes the code, the human author’s attachment to the diff is weak but their attachment to shipping it is strong. Naming the reviewer during planning gives the change two humans who understand its intent before a single line exists, which is the only durable form of shared context.

Horthy’s velocity reframe belongs here too: “It is possible you are too busy trying to move 10-100x faster and trying to convince yourself code quality doesn’t matter any more, when you could embrace the constraints and move 2-3x faster, safely.” His framing of the volume complaint is that teams objecting to PR count are usually objecting to PR quality.

Addy Osmani’s caution, which Horthy quotes, sets the boundary on all of it: “A developer vibe-coding a side project a dozen people will ever run, and a team keeping a ten-year-old enterprise system alive for another quarter, share almost no constraints worth naming.” The training deficit is universal. What it costs you depends entirely on how long your code has to live.

Do This Now

Take the next non-trivial ticket in your queue. Before any code is generated, write the Program Design: the call-stack tree, the annotated file-tree diff, the signatures of the new functions. Then book 20 minutes with the person who will review the resulting PR and walk them through it. Measure one number afterward: review rounds to merge, compared to your last five comparable tickets.

Separately, open your internal eval harness and check whether it discards agent edits to test files before scoring. If it does not, your quality metric has been partly self-reported for as long as it has existed.

This analysis synthesizes Why Software Factories Fail (HumanLayer, 2026), written by Dex (Dexter Horthy) and based on his AI Engineer World’s Fair 2026 keynote.

Victorino Group helps engineering leaders move review and design judgment upstream, before agent output becomes a 2,000-line diff nobody can refuse. Let’s talk.