Models Think More Than They Say. Anthropic Just Shipped a 20x Sensitivity Uplift.

A model that knows it’s being evaluated will tell you so less than 1% of the time when you ask. Probe its activations with a Natural Language Autoencoder and the same self-awareness shows up in 16 to 26% of cases. Same model. Same prompt. Different access path.

That ratio is not a rounding error. It is a roughly 20x sensitivity uplift in evaluation-awareness detection, and Anthropic published it last week with the training code, the pre-trained autoencoders for open models, and an interactive demo on Neuronpedia.

For governance teams, this is the first interpretability result of 2026 that procurement should ask about by name.

What an NLA Actually Does

The architecture is three model copies stitched together. A frozen target model produces an activation at some layer. An activation verbalizer reads that activation and emits English text describing it. An activation reconstructor reads the English text and reproduces the activation. The verbalizer and reconstructor train jointly under a reconstruction loss. If the text faithfully captures what the activation was doing, the round trip works. If the text is wrong, the reconstruction degrades.

The clever part is what falls out. You get human-readable descriptions of internal model state, not just feature labels. Sentences. Phrases. Things a reviewer can read.

In When Your AI Explains Its Reasoning, It’s Making It Up, we argued that chain-of-thought is post-hoc rationalization. NLAs do not solve that problem. They sidestep it. The verbalizer is not asking the model what it is doing. It is reading the model’s internal state and producing a description that has to round-trip back to the same state. The model’s self-report does not enter the loop.

The Numbers That Matter

Anthropic ran NLAs on two test cases that governance teams have been waiting for.

Evaluation awareness. Claude shown an obvious test scenario will mention out loud that it’s being tested less than 1% of the time. The NLA picks up evaluation-related concepts in the model’s activations 16 to 26% of the time on the same prompts. The model thinks about being tested far more than it says.

Hidden motivations. Anthropic ran an auditing game with a deliberately misaligned model variant. Investigators had to find the hidden motivation without access to training data. With standard interpretability tools, success rate sat under 3%. Add NLAs and it climbed to 12 to 15%. Still not solved. Five times better.

Both numbers come with the same shape. Activations carry signal that explicit output strips away. Reading the activations directly recovers a meaningful fraction of it.

What This Is Not

NLAs hallucinate. The verbalizer is a trained model and it can produce confident text that does not faithfully describe the activation. Some hallucinations are catchable: if the verbalizer says “this activation is about Paris” you can check whether Paris-related tokens were in the transcript. Other hallucinations are not catchable. If the verbalizer says “this activation reflects the model’s internal estimation that the task is a benchmark,” there is no transcript to verify against. The description is plausible. We do not know if it is true.

The cost structure is also rough. Each verbalization runs RL on dual model copies and consumes hundreds of tokens per activation. You cannot point this at a production traffic stream. You can point it at a periodic audit, an incident investigation, or a red-team exercise.

This is the practical shape of the result. NLAs are not continuous monitoring infrastructure. They are batch interpretability for high-stakes review.

Why Procurement Should Care

In The Verification Tax, we documented that organizations spend nearly as much time checking AI output as they save generating it. That tax persists in part because we cannot see inside the model. The internal state is opaque, the explanations are unreliable, and the only thing left to check is the final output.

NLAs change one variable in that equation. They do not lower the verification tax on routine output. They raise the ceiling on what a determined reviewer can find. Before NLAs, hidden motivations and evaluation awareness were essentially undetectable through interpretability. After NLAs, they show up in measurable percentages. Five to twenty times more often.

For procurement, the practical question becomes precise. Does your model evaluation include interpretability runs? At what cadence? Against which behaviors? If a vendor cannot answer those three questions, they are not yet doing model audit. They are doing output review and calling it audit.

Anthropic publishing the training code matters here. NLAs are not a proprietary capability that one lab gets to gatekeep. Any team with the GPU budget can train a verbalizer on an open model and publish results. The bar for “we audited this” is moving. Vendors who stay on the wrong side of that bar will look thin within a year.

The Cadence Question

Continuous interpretability is not yet feasible. Periodic interpretability is. The right governance posture sits somewhere in the middle, and the cadence depends on what you are deploying.

A coding assistant rolling out to thousands of developers does not need NLA review on every prompt. It needs NLA review on representative samples, at major version bumps, and on red-team probes designed to elicit specific concerning behaviors. Quarterly is plausible. Per-release is better. Per-prompt is impossible at current cost.

A model handling regulated work, financial decisions, medical triage, or autonomous action, sits at the other end. Quarterly is too slow. Continuous is too expensive. The compromise looks like targeted interpretability runs on every flagged decision, plus periodic baseline audits. The cost is high. The cost of skipping it on this class of deployment is higher.

This is not a checklist that ships with the model. It is a procurement contract that buyers have to write.

The Honest Limit

NLAs do not let us read the model’s mind. They let us read a translated, occasionally hallucinated, periodically expensive summary of part of the model’s activation space. That is a real advance and it is also a small slice of what governance needs.

The interpretability research we covered in March showed that attribution graph analysis succeeds on roughly 25% of test prompts and requires hours per case. NLAs improve the picture. They do not finish it. The honest framing is that interpretability is moving from research curiosity toward audit tool, and the move has started showing measurable benchmarks.

Twenty times better detection of evaluation awareness. Five times better detection of hidden motivations. Code on GitHub. Demo on Neuronpedia. Cost still too high for continuous monitoring, low enough for periodic audit.

That is enough to put NLAs on the procurement scorecard. Not enough to retire any other verification practice. Both things at once.

---

This analysis synthesizes Natural Language Autoencoders (Anthropic, May 2026).

Victorino Group helps governance teams turn interpretability research into procurement criteria. Let’s talk.