Your Taxonomy Is Now the Agent's Permission Model

Enterprises spent $37 billion on generative AI in 2025, up from $11.5 billion the year before, and only 15% report generative AI deployed at enterprise scale. Those two numbers come from the same body of research: Menlo Ventures on the spend, the World Quality Report 2025 on the deployment rate. Between the money and the scale sits a category of failure that gets attributed to the model and belongs to the store the model reads from.

Patrick Neeman, an AI UX leader at Workday, made the argument in UX Collective in July 2026, and his framing is worth quoting once because it is the cleanest version of it: “You don’t have an AI problem. You have an information architecture problem wearing an AI costume.”

Retrieval optimizes for surface similarity

Retrieval-augmented generation, as described by Lewis and colleagues in 2020, does something narrower than most buyers assume. It embeds a query, searches a vector store, and returns passages that sit close to the query in that space. Closeness is lexical and semantic proximity. Neeman’s description of the consequence: retrieval “finds the loudest match, the one that shares the most surface words with the query, regardless of whether it is current, authoritative, or true.”

The store decides what qualifies as loud. If a 2019 policy PDF, its 2024 replacement, and a Slack thread where someone paraphrased both are all sitting in the same index with no type, no effective date, and no authority label, the retriever has no basis for preferring the one that governs. It will hand back whichever chunk shares the most vocabulary with the question. A model then writes a fluent answer grounded in the wrong passage, and the fluency is what makes it dangerous. Language models “don’t fill gaps with judgment; they pattern-match over the mess and reproduce it at scale, confidently.”

Anthropic’s contextual retrieval work gives a rough sense of how much of the failure is structural rather than model-side. Prepending explanatory context to each chunk before indexing cut failed retrievals by up to 49%. Same model, same query, better-described content, half the misses. Nobody swapped the reasoning engine.

The escalation nobody priced in

Retrieval quality as an information architecture problem is a known argument. Here is the part that changed underneath it.

An assistant that answers questions produces text a human reads and can discard. An agent that files a claim, adjusts a record, issues a refund, or provisions access produces state. Neeman puts the difference plainly: “A wrong answer erodes trust. A wrong action moves money, changes records, and triggers systems downstream that assume it was correct.”

Now trace how an agent decides what it may act on. It looks up the entity, resolves what that entity is related to, and applies whatever rules it retrieves about scope. Every one of those steps is a lookup against the same untyped, unlabeled store. If two customer records share a name, if a deprecated policy lives beside its replacement, if a document has no owner attached, the agent resolves the ambiguity the same way the retriever does: by surface similarity. Which means your taxonomy is now deciding what the agent is allowed to touch.

Call it a taxonomy, an ontology, or a semantic layer. The job is identical: telling a machine what a thing is and how it relates to other things. Under an assistant, that was a filing system. Under an agent, it is an authorization surface, and it was never designed to hold that weight. As Neeman puts it: “The mess didn’t change. The blast radius did.”

Two courts, one conclusion

In Moffatt v. Air Canada, a British Columbia tribunal held the airline liable for a bereavement-fare answer its chatbot got wrong. The airline argued the chatbot was “a separate entity responsible for its own words.” The tribunal rejected that. Air Canada owned the output because Air Canada deployed the system.

We covered a German court reaching the same conclusion in a separate case on AI output liability. Two jurisdictions, different facts, one rule: the operator owns what the system says. Neither court asked which model was used, which vendor supplied it, or what the temperature setting was. Liability attached to the party that put the system in front of a customer.

Extend that to agents and the exposure changes shape. A wrong sentence produced a refund and a bad news cycle. A wrong action produces a transaction, an audit trail that records it as intentional, and downstream systems that already consumed it. When the regulator or the plaintiff asks why the agent did that, “the model hallucinated” is not an answer. The answer will be a retrieval trace, and it will point at your content store.

Run this on one wrong answer today

The diagnostic Neeman offers costs an afternoon and does not require a platform decision. Take one wrong answer your AI produced recently. Pull the passages it retrieved to produce it.

You will land in one of two places. Either the retrieved passages were correct and the model reasoned badly, which is a model problem worth solving with a model. Or the retrieved passages were stale, ambiguous, duplicated, or drawn from a source with no authority, which means the model summarized exactly what you gave it. You cannot tell those apart from the output alone. Both look like a confident wrong answer. Only the retrieval trace separates them, and most teams buy a better model without ever looking.

Run the sample large enough to be honest, twenty answers rather than one, and you get a ratio. That ratio is the argument for where the next dollar goes.

The five moves that follow, adapted from Neeman’s pre-purchase audit:

1. Audit what your AI retrieves from, not only how it answers. Evaluation suites score responses. Score the corpus.
2. Type your content before you index it. Document type, effective date, owner, authority level, supersession status. Untyped content cannot be filtered, and what cannot be filtered gets retrieved.
3. Fix labels and taxonomy before you fix prompts. Prompt engineering compensates for a bad store one query at a time and never compounds.
4. Give agents an explicit model of relationships and permissions. What belongs to what, what depends on what, and what an agent is allowed to touch. Written down, not inferred at runtime from embeddings.
5. Put an information architecture cost on an AI outcome. Neeman’s version: “Simply said, poor information architecture can now be measured in token costs. A lot of them.” Retries, longer contexts, and human rework are the invoice.

The related discipline is traceability. We argued in governance you cannot trace that a decision you cannot connect to the code implementing it is not governed, and in the agent diary that agent actions need a durable record. Retrieval traces are the same instrument pointed at the input side.

Google’s AI told people to put glue on pizza in 2024 because a joke post looked, to a retriever, like a recipe. That was funny while the output was a sentence. Give the same retrieval stack a purchase order and a service account, and the joke post becomes an authorization decision. Before you approve the next model upgrade, ask what your agent retrieved last time it was wrong. If nobody can produce that trace, the store is where your budget belongs.

---

This analysis synthesizes Information Architecture Is the Foundation Artificial Intelligence Is Starving For (Patrick Neeman, UX Collective, July 2026).

Victorino Group helps teams audit the content stores their agents act on before those stores become a permission model by accident. Let’s talk.