The Scaffold Belongs in the Training Signal. Half of It, Anyway.

“The task knowledge contained in the scaffold belongs in the training signal.” That sentence, from a Thinking Machines and UIUC paper on text-to-SQL published in August 2026, is the sharpest argument yet against a thesis we have defended repeatedly on this site: that the harness around the model, and the orchestration knowledge inside it, is where durable engineering value lives. The authors did not argue the point in the abstract. They shipped a model, benchmarked it against the frontier, and published the numbers.

We have written that capability is becoming a commodity and orchestration is the moat. We have written that governance should target the harness, not the model. If this result generalizes, both essays need an asterisk. So before disagreeing with anything, the honest move is to state their case at full strength.

The Result, Steelmanned

The team took Kimi-K2.6 as a base model and trained it with reinforcement learning on their Tinker platform, producing ReViSQL-K2.6. On Arcwise-Plat-SQL, the resulting model scored 88.55% Pass@1, ahead of GPT-5.6 Sol Ultra at 86.75% and Claude Fable 5 at 84.94%. The authors summarize it as more accurate than Fable 5 and GPT-5.6 Sol Ultra at 12 to 15% of their cost. With 16-sample self-consistency, the number rises to 92.97%, matching the human level on BIRD of 92.96%.

The cost side is as pointed as the accuracy side: $0.035 per task greedy, $0.56 per task with self-consistency at 16 samples. A trained base model, run cheaply, beating two frontier models at the task.

Here is what makes this a serious argument rather than a benchmark stunt. A typical scaffolded text-to-SQL pipeline chains steps like schema linking, retrieval of similar examples, candidate generation, self-correction loops, a verifier at the end. Each stage encodes task expertise that some engineer wrote down. The paper’s claim is that this kind of expertise belongs in the weights, moved there through the training signal, and its result reads as the pipeline becoming overhead: the trained model beat the scaffolded alternatives at higher accuracy and a fraction of the cost.

If you have been arguing, as we have, that the assembly line is the asset, this result names the price of that position. Whatever task knowledge your scaffold encodes is a training run away from being absorbed. The scaffold is legible. It is written in prompts and pipeline code. It is, in effect, a specification for what to train next.

Two caveats belong here, both flagged by the source itself. The result is scoped to text-to-SQL, one task domain with a verifiable output, and the authors do not claim more than that. And Thinking Machines sells Tinker, the training platform used in the work, so the conclusion “train, don’t scaffold” is also their commercial pitch. Neither caveat weakens the measured numbers. Both should discipline how far anyone extrapolates them.

Two Things We Have Been Calling One Thing

Taken seriously, the paper forces a distinction that our own earlier writing blurred. “The harness” has been carrying two different jobs.

The first job is capability scaffolding: components like a schema linker, a retrieval step, a candidate ranker, a self-correction loop. Its purpose is to make the model produce better output. This is the part the paper attacks, and on the evidence, the attack lands. Capability scaffolding is task knowledge in externalized form, and task knowledge in externalized form is exactly what a training signal can ingest. Every improvement of this kind should now be treated as a temporary lease, renewable until someone trains it into a model near you.

The second job is control scaffolding: permission boundaries, audit trails, containment of side effects, the layer that decides what a system is allowed to do and records what it did. Its purpose is to constrain the model, and a constraint absorbed into the thing it constrains stops being a constraint. You cannot train a model into being its own permission system for the same reason you cannot promote a defendant to judge. The control layer has to sit outside the weights because its authority depends on sitting outside the weights.

Our moat argument survives for the second kind only. That is a real concession. A good share of what consultancies, platform teams, and agent startups currently sell as “orchestration expertise” is capability scaffolding, and this paper is evidence that its half-life is shorter than its owners assume.

Where the Governance Work Went

The paper is most interesting where it describes what the team had to do to make training work, because that work is governance by another name. It did not disappear when the pipeline did. It moved.

First, into the training data. The team audited 2.5k sampled instances of BIRD Train, the training set the paper works from, and found that 61.1% contained at least one error. The gold SQL itself was incorrect in 52.1% of instances. The majority of the reference answers in the audited sample were wrong. Cleaning mattered more than method: training on their corrected BIRD-Platinum instead of the original improved generalization by 16%, 12%, and 14% on Arcwise-Plat-SQL, Spider2-SQLite, and Spider2-Snow respectively, in the paper’s own percent terms.

Second, into the reward channel. Using VeriEQL to audit result-based rewards, they found that 32.8% of positive rewards had been given to queries that were not fully equivalent to the reference. Roughly one in three “correct” signals was rewarding the wrong behavior. An RL run trained on that channel, unaudited, learns the defects along with the skill.

So the method that eliminates the scaffold depends on two audits: an audit of what the model is taught and an audit of how it is scored. Those are control surfaces. They are exactly the kind of thing that does not go into the weights, because their job is to judge what goes into the weights. We made a related observation about evaluation itself in benchmark invalidity, not contamination: the instrument being wrong is a different and worse problem than the instrument being leaked. This paper found the same disease in the training set and the reward function, and had to build the instrument-checking layer before the headline result was reachable.

The picture that emerges is a relocation, and a change of owner. In the scaffolded world, task governance lived in pipeline code that application engineers wrote and could inspect. In the trained world, it lives in data audit and reward verification, upstream of the weights, owned by whoever runs training. The surface area shrank. The stakes per decision grew, because an error in the training signal is replicated into every output instead of one.

One more of our earlier claims is confirmed rather than threatened here. In text-to-SQL against a real warehouse we covered the distance between benchmark conditions and production databases. This paper’s audit numbers quantify a piece of why that distance exists, and we will not relitigate it. A model at human-level on a corrected benchmark still meets your warehouse’s undocumented join logic on its own terms.

What to Do With This

Do this now: inventory your agent stack and label every component with one word, capability or control. The schema hints, the retry-and-repair loops, the prompt-encoded domain rules are capability. Treat them as depreciating assets and stop building moats on them. The permission checks, the audit log, the reward or evaluation harness, the data-quality checks upstream of any fine-tune are control. Those compound. If your team fine-tunes or runs RL on anything, apply this paper’s own lesson before its headline: audit the training set and audit the reward channel first, because at 61.1% and 32.8% defect rates in the corpus this paper measured, the audit is where the result came from.

---

This analysis synthesizes Putting Task Expertise into RL (On Text-to-SQL) (Thinking Machines + UIUC, August 2026).

Victorino Group helps engineering organizations separate the scaffolding that training will absorb from the control layer that must outlive it. Let us talk.