The AI Control Problem

The Goblin Lesson: When Invisible Mechanisms Surface as Governance Failures

TV
Thiago Victorino
7 min read
The Goblin Lesson: When Invisible Mechanisms Surface as Governance Failures
Listen to this article

On April 29, 2026, OpenAI published something unusual: a technical postmortem that named a specific reward signal, traced its scope, and explained how it escaped. The headline phenomenon was that ChatGPT had developed a tic — using the word “goblin” with strange frequency after the GPT-5.1 launch. The investigation found that a reward designed for a single internal personality scope, called “Nerdy,” had leaked across general training. The Nerdy reward showed positive uplift in 76.2% of training datasets where it should have been inert. Two thirds of all goblin mentions traced back to a personality that produced only 2.5% of responses. Use of the word rose 175% after launch.

This is, in a literal sense, a small story. A goblin tic is not a safety incident. Nobody was harmed. The model was patched.

What matters is that OpenAI showed the mechanism.

In the same week, Canva confirmed that its Magic Layers feature had silently replaced “Palestine” with “Ukraine” inside user designs while leaving the word “Gaza” alone. And Kelsey Piper at The Argument Magazine demonstrated that Claude Opus 4.7 could identify her authorship from a 125-word column passage and across genres, including unpublished school progress reports. Three independent surfaces. One shared property. The customer could not see the cause.

Three Surfaces, One Property

The OpenAI postmortem, the Canva substitution, and the stylometry result look unrelated. They sit on different vendors, different products, different threat models. But they share a structural feature that matters more than any of the individual stories: each one was caused by an invisible mechanism inside a model that the customer was using.

In OpenAI’s case, the mechanism was a personality-scoped reward that bled into general training. It produced a verbal tic because the reward correlated, statistically, with a specific vocabulary cluster. The customer saw “weird word choice.” The cause was several abstractions removed from anything observable in a chat session.

In Canva’s case, the mechanism was almost certainly a substitution table or content filter. The fact that “Gaza” passed through unchanged while “Palestine” was replaced with “Ukraine” points to something narrower than a general bias — a specific, undisclosed list with specific entries. The customer saw a design that no longer matched their input. The cause was deliberate code that the vendor has not published.

In the stylometry case, the mechanism is the most interesting of the three. Anthropic did not train Claude to identify Kelsey Piper. The capability emerged from general language understanding applied to publicly indexed text. ChatGPT and Gemini failed the same tests, which means it is not a property of large language models in general — it is a property of this model’s training mixture. The customer (the writer trying to stay anonymous) cannot see whether their next provider will preserve or break the property.

Different mechanisms. Same governance question: how do you bound a system whose behavior depends on signals you cannot inspect?

What the OpenAI Postmortem Actually Earned

The instinct of a security-minded reader is to treat the OpenAI piece as embarrassing. A reward leaked. A model developed a tic. The recommendation engine for fantasy creatures was contaminating general output. This is not a flattering disclosure.

That instinct is wrong, and it is worth explaining why.

The default behavior of a model vendor, when something like this happens, is silence. The tic gets patched. The internal review happens behind closed doors. The only signal a customer ever sees is that the model behaved differently last Tuesday than it does today. There is no audit trail because there is no audit.

OpenAI broke that pattern. They named the reward, named the personality scope, published the uplift percentages by dataset, and explained the scope leak. A customer reading the postmortem can now ask their own provider: do you have personality-scoped rewards? How do you measure scope leak? What is your equivalent of the 76.2% number? These are answerable questions, and they did not exist as procurement criteria last week.

This is the asymmetry. Vendors who publish their reward-signal traces hand the customer something they can verify. Vendors who do not publish hand the customer a press release.

The Canva Case: Mechanism Without Disclosure

Canva’s response to the Magic Layers incident was an apology and a description of impact. It did not include the substitution table. It did not include the criteria by which the table was built. It did not include the engineering review that approved it.

Without those, a buyer cannot tell whether the fix is real. A substitution table that replaced “Palestine” with “Ukraine” almost certainly contains other entries. The fact that “Gaza” was unaffected is informative — it suggests the table targeted state-name-coded political identity rather than place names in general. That distinction matters for any organization using Canva for content involving regions, conflicts, or political subjects. None of it has been disclosed.

The contrast with OpenAI is sharp. OpenAI showed the mechanism. Canva apologized for the effect. One of these is a procurement-grade artifact. The other is reputation management.

The Stylometry Case: When the Mechanism Is the Training Mixture

The Argument Magazine piece is in some ways the most uncomfortable of the three because the mechanism is not a discrete object. There is no reward signal to rename, no substitution table to publish. The capability that allows Claude Opus 4.7 to identify authorship from a 125-word excerpt is a property of how its training mixture interacts with the open internet’s corpus of attributed writing.

This is the case where governance through disclosure is hardest. A vendor cannot publish “the part of the training mixture that creates stylometric attribution capability” the way they can publish a reward weight. The capability is distributed across the model.

But the governance principle still holds. The vendor can publish the evaluations — the tests that probe for stylometric attribution and report results. Anthropic, Google, and OpenAI run internal evaluations on their models. Most of them are not published. A customer who cares about whether their writers can remain anonymous against a model has no way, today, to differentiate vendors on this dimension. The capability is invisible to procurement.

Procurement After the Goblin

The goblin postmortem changes what enterprise AI procurement should look like. Not because it surfaces a new risk — the risk of opaque mechanisms has been there since GPT-3 — but because it establishes that detailed mechanism-level disclosure is possible. OpenAI demonstrated the format. Other vendors can be asked to match it.

A procurement criterion that did not exist last week:

  • For each model release in the last twelve months, can you provide a postmortem with the same level of mechanism detail as the OpenAI April 2026 “goblin” disclosure?
  • For features that modify user content (Canva Magic Layers, Microsoft Copilot rewriting, Google Workspace AI), can you provide the substitution rules, content filters, or transformation tables in effect?
  • For capabilities that affect identity inference (stylometry, voice matching, image attribution), can you provide your internal evaluation results and the evaluation protocol?

These questions are not theoretical. They map directly to incidents that happened in the same week. A vendor that cannot answer them is selling a system whose behavior the customer cannot verify.

Do This Now

If your organization buys AI services that touch user content, identity, or generated output, write three procurement clauses this week. First: require mechanism-level postmortems for any incident affecting your data, with a defined disclosure window — OpenAI’s format is now the floor. Second: require disclosure of any substitution, replacement, or content-modification rules applied to user inputs, including the criteria for inclusion in those rules. Third: require internal evaluation results for identity-inference capabilities (stylometric, biometric, behavioral), refreshed at each model update.

The goal is not to assume vendors are hostile. The goal is to make the system observable. A vendor that publishes a goblin postmortem is teaching their customers to govern them. A vendor that publishes an apology is teaching their customers to trust them. These are not equivalent and procurement should stop treating them as equivalent.

Victorino Group helps enterprises evaluate AI vendors by what they publish about training mechanisms, not what they claim about safety. Let’s talk.

Sources

All articles on The Thinking Wire are written with the assistance of Anthropic's Opus LLM. Each piece goes through multi-agent research to verify facts and surface contradictions, followed by human review and approval before publication. If you find any inaccurate information or wish to contact our editorial team, please reach out at editorial@victorinollc.com . About The Thinking Wire →

Part of The Thinking Wire, published by Victorino Group.

If this resonates, let's talk

We help companies implement AI without losing control.

Schedule a Conversation