Prompt Debt: How Hand-Tuned Prompts Lock You to a Dying Model

Phrase ten medical questions in a patient’s voice and Claude Opus declined every one. Rephrase the identical facts in a physician’s voice and it answered all ten. The model weights did not change. The words around them did. That single flip, from a study David Breunig cites in his June essay, is the whole problem with how most teams specify agent behavior: they write it in prose, and prose is bound to the model that happened to be running when they wrote it.

David Breunig gave the cost a name worth keeping: prompt debt. It is the accumulating liability of specifying behavior in natural language. Every edge case you patch with another sentence, every refusal you coax away with a softer phrasing, every “always do X unless Y” clause makes the prompt longer, harder for a teammate to read, and more tightly coupled to one model’s quirks. The debt is invisible on the day you write it. It comes due on the day you try to upgrade.

What Prompt Debt Actually Is

A prompt starts clean. One paragraph, one job. Then production happens. The agent mishandles an unusual invoice, so someone adds a clause. It refuses a legitimate request, so someone adds an exception. It hallucinates a field name, so someone adds a warning in bold. Six months later the prompt is two thousand tokens of accreted special cases, and nobody on the team can say which sentence is load-bearing and which is vestigial. Change one line and three behaviors shift in ways no one predicted.

That is technical debt by any honest definition. It fragments the team, because the prompt becomes a private artifact only its last editor understands. It resists refactoring, because there is no test that says whether a rewrite preserved behavior. And it has a property ordinary code debt does not: it is welded to a specific set of model weights.

The Prose Is Welded to the Weights

The studies Breunig cites make the coupling concrete, and they are independently citable, so you do not have to take the commentary on faith.

The patient-versus-physician result (arxiv 2604.07709) is the clean one. Same clinical facts, two framings, opposite outcomes: ten refusals became ten answers. The behavior the prompt author wanted lived in the phrasing, not in any stable property of the model.

A Harvard group went further (arxiv 2407.06866v3). They showed that statements with no logical bearing on the request, a stated preference for a sports team, for instance, shifted how often the model refused. The decision boundary the prompt author is implicitly tuning against is sensitive to noise that has nothing to do with the task. You are not writing a specification. You are fitting a curve to one model’s surface, by hand, one phrase at a time.

When the next model ships, that surface moves. The phrasings you tuned against the old weights now land differently. Refusals reappear. Edge cases you thought you had closed reopen. The prompt that worked on Friday produces different behavior on the new model Monday, and you have no measurement that tells you what broke or by how much.

Why Enterprises Freeze on a Depreciating Asset

So they do the rational short-term thing: they stop upgrading. A Berkeley analysis Breunig points to (arxiv 2512.04123) documents enterprises holding onto older models specifically because upgrades break their agents. The evidence is visible in the aggregate. As of Datadog’s State of AI Engineering report (March 2026), GPT-4o was still the most-used model in production, long after newer and stronger options were available.

Read that as a balance-sheet fact. The organization has accumulated so much prompt debt that the cost of migrating it exceeds the benefit of a better model. The prompts are an asset that only works against weights that are themselves depreciating. The model vendor moves on, deprecation notices arrive, prices on the old endpoint creep up or the endpoint sunsets entirely, and the team is stranded on a version it cannot leave without re-tuning hundreds of brittle prose clauses by hand.

This is the part a CFO understands immediately even if the term “prompt” means nothing to them. You have capital tied up in an asset that is losing value, you cannot redeploy it without a large re-engineering cost, and the lock-in is to a third party’s release schedule. Un-measured prompts are a silent liability that converts every model improvement from an opportunity into a migration bill.

The Exit Is Measurement, Not Better Prose

The way out is to stop encoding the specification in the prose and start encoding it in evaluations. State what correct behavior looks like as a set of cases with expected outcomes, then measure any model, any prompt, against that set. The eval becomes the specification. The prompt becomes a replaceable implementation detail.

This decouples the spec from the weights. When a new model ships, you do not read two thousand tokens of prose and guess. You run the eval suite and read a number. If the new model scores higher, you upgrade and delete prompt clauses the better model no longer needs. If it scores lower on three cases, you know exactly which three and can decide with data instead of dread. The migration that used to be a leap of faith becomes a regression test.

We have argued the mechanics of this elsewhere and will not repeat them here. Evals are the deterministic shell around a probabilistic model. Much of what teams cram into prompts belongs in passive context the model reads as state rather than instructions it must obey. And the discipline underneath all of it is verification before trust: a behavior you have not measured is a behavior you do not actually control, regardless of how carefully you worded the prompt.

The economic reframing is the new part. Evals do more than protect quality. They keep the specification portable across models, which is how you keep the right to upgrade. A team with a strong eval suite treats models as interchangeable parts and shops for the best one every quarter. A team with prompt debt and no evals is a hostage to whichever model it tuned against first.

Do This Now

Pick your most business-critical agent, the one a customer or a regulator would notice if it misbehaved. Open its prompt and count the clauses that exist only to patch a specific past failure. That count is your prompt debt, made visible for the first time.

Then convert the ten failures you care about most into ten evals: input, expected outcome, pass or fail. Run them against your current model to set a baseline. Run them against one model you have been afraid to upgrade to. The delta tells you, in one number per case, exactly what an upgrade would cost or save. You have just turned a paralyzing migration into a decision you can defend in a budget meeting.

Do this before the vendor deprecates the model you are standing on. The prompts you cannot measure are the ones quietly setting your model strategy for you.

---

This analysis synthesizes The Problem is Prompt Debt (David Breunig, June 2026). The underlying studies are independently citable: arxiv 2604.07709, 2407.06866v3, and 2512.04123.

Victorino Group helps engineering leaders replace brittle prompts with eval-anchored specifications that survive model upgrades. Let’s talk.