Refactoring Now Has a Measured Return, and It Is 83%

An identical change prompt cost 159,564 input tokens before refactoring and 27,360 after. Fifteen refactoring steps in between. The line count of the code being changed went from 17,155 to 16,608, which is to say it stayed put. Giles Edwards-Alexander, CTO for Europe, Middle East and India at Thoughtworks, ran that measurement on a roughly 150,000-line application written almost entirely by agents, and published it in Martin Fowler’s Exploring Gen AI series. His summary of the mechanism is the sentence worth keeping: the saving happens “because the agent has to read less code. But it is not because there is less code to read.”

That reframes refactoring from a craft preference into a line item. Every future change to that module now bills 83% less on input.

The Protocol Is the Contribution

The number matters less than how it was obtained, because the how is reproducible on your repo next week.

The application is about 120,000 lines of Rust plus TypeScript and Terraform, built by Claude Code with some Cursor, maintained by a single developer. Its data access layer had grown past 6,000 lines and eventually reached 17,155, all in one Rust file. Edwards-Alexander picked a representative change, then repeated this loop after each refactoring step: spawn a fresh sub-agent, hand it the same prompt, record the input tokens it consumed to make the change, discard the sub-agent.

The discard is the whole experiment. His framing is exact: “Precisely because agents never learn this was now possible to run as an experiment.” A human engineer measured this way would carry knowledge of the codebase from step one into step fifteen and contaminate every subsequent reading. The agent’s amnesia, usually the thing we complain about, is what makes the measurement clean. Same prompt, same task, no carry-over, fifteen data points.

Two variables get tracked per step: input tokens for the representative change, and lines in the largest single file. That second one is the operable metric, and most teams leave it untracked.

The Curve Has a Knee

The savings accrue unevenly across the fifteen steps, and that is the finding a manager needs before approving the work.

Input tokens hovered between roughly 131,000 and 171,000 for the first eleven steps. Eleven steps of real refactoring work with essentially nothing to show on the meter. At step twelve, with the largest file down to 9,269 lines, tokens fell to 104,080. At step fifteen, largest file at 3,695 lines, they hit 27,360. Edwards-Alexander describes the shape as staying “fairly flat until the largest file starts to fall, and then they drop before, in the words of Claude, falling off a cliff.”

Anyone who evaluates a refactoring initiative after two weeks and sees a flat token line will kill it one step before the payoff. The largest-file line count moves first. The token bill reacts later. Track the leading indicator or you will terminate the work while staring at the lagging one.

The endpoint was 19 Rust files, the biggest of which is a 3,695-line test library. A modest decomposition by any standard, and a file that would still make a reviewer wince. Small enough, though, that the agent stopped dragging an entire module into context to change one function.

The Dollar Figure Is Deliberately Unimpressive

At the $3 per million input tokens Sonnet 5 charged at the time, the 132,204 tokens saved come to 39.7 cents. The author says so plainly: “Not a lot.”

Presenting it any other way would be dishonest, and the honesty is what makes the argument land. One change saves pocket change. The claim is about recurrence: “that saving is not a one-off. Every single change that touches the data access layer from this point forward now costs significantly less.” Multiply 40 cents by every future edit to a core module, then remember output tokens run five times the input price and that a smaller context also cuts latency and error rate. The economics stop being about the cents and start being about a permanent change in the slope.

The cost side gets the same treatment. Roughly eight hours, mostly unattended, one human intervention after six hours forty minutes. The tokens burned doing the refactoring were not counted, though he bounds them: “The upper bound is five million.” So the investment is real and the payback is a function of how often that module gets touched. For a data access layer, that is constantly. For a file everyone leaves alone, refactoring buys zero, which is a useful thing to be able to say with numbers.

What Claude Could Not Do

The agent did not choose the refactorings. Edwards-Alexander is blunt: “Claude was not good at refactoring” and “is unable to look at code, look at refactorings in general and work out which are suitable to apply: a human needs to actively guide it.” The plan followed named Fowler moves, Extract Class, Extract Function, Replace Inline Code with Function Call, Move Function, selected by a person who knew which one fit which mess.

There is a mechanical detail that should worry anyone planning to automate this. The actual edits were performed by Python scripts using grep and sed, and those scripts “frequently got confused by indentation. Oh, the irony.” Coding agents writing brittle text-manipulation scripts to restructure code they could have restructured semantically.

He also flags that “randomly cutting the file into smaller files is unlikely to help as much,” and that he applied “a stricter refactoring than most human engineers would follow.” Splitting on line count alone would fail to reproduce this curve. Coherent boundaries are what let the agent read one file instead of nineteen.

The Gate That Did Not Fire

Buried in the write-up is a failure that sits outside code quality entirely. The harness building this application already had an explicit refactoring step. It ran. It “did not prompt Claude into improving this file” while that file grew to 17,155 lines.

An automated quality control existed, was wired in, executed on schedule, and stayed silent while the exact condition it was meant to catch got twenty times worse. A human eventually noticed the file size. That is a control-plane failure, and it is the more transferable lesson, because most teams adding agent quality gates are adding gates that report success by staying quiet. Silence from a gate is evidence the gate ran. Treating it as evidence the gate works is where the twenty-fold growth hides.

We have argued before that agent-written code accumulates debt everyone leaves unswept, and that context structure drives token cost. This experiment supplies the meter for both, and adds a third: the automated sweeper can be present and inert.

The Honest Limits

One experiment, one greenfield application, one developer. The author states this himself and the edge deserves to stay sharp. Token counts are approximations, computed with tiktoken by dividing character count by four, because Claude “doesn’t provide reliable methods for counting tokens live.” The 83% is a well-instrumented single case. Treating the figure as a benchmark stretches it past what it carries. What survives replication is unknown until someone replicates it, and the protocol is cheap enough that they should.

Do This Now

Measure the largest single file in the module your agents edit most. If it is over 5,000 lines, you have a candidate.

Pick one representative change, the kind of edit that lands on that module every week. Run it in a fresh sub-agent, record the input tokens, throw the sub-agent away. That is your baseline. Then apply named refactorings with human judgment about which ones fit, and re-run the identical prompt in a fresh sub-agent after each step. Plot input tokens against largest-file line count and stop when the curve flattens at the bottom.

Two rules keep the exercise honest. Expect the token line to sit flat for the first several steps, and treat a quality gate that has been quiet all month as unverified. Go look at the file size yourself.

This analysis synthesizes The Economic Benefit of Refactoring (Giles Edwards-Alexander, Thoughtworks, July 2026).

Victorino Group helps teams instrument the recurring token cost of their codebases and turn refactoring into an investment with a measured return. Let’s talk.