If You Can't Verify Your AI's Output, Neither Can Your Users

When an AI feature is hard to evaluate, most teams respond by building a bigger eval harness. More test cases, more graders, more offline runs. Hamel Husain, who has spent three years building AI evals for a living, argues the difficulty is pointing somewhere else. “It’s hard to eval” is usually a product design smell. If you cannot easily verify the output, the problem lives in how the artifact is built, and no amount of downstream measurement will fix that.

That reframe matters because it moves the work upstream, from a metrics problem the ML team owns to a design problem the whole product owns.

Verification Used to Be Free

For most of software history, verification happened incidentally. You wrote the code and you read the code as you wrote it. You drafted the report and you understood every sentence because you produced each one. Checking was woven into making. It rarely showed up as a separate line of work because it was never a separate step.

AI breaks that. The output arrives whole, produced by a system whose reasoning you did not follow. Now verification is a distinct act, performed after the fact, on an artifact you did not build. That is the shift Husain names: with AI, verification stops being a byproduct of creation and becomes the bottleneck of the workflow.

We have made the financial version of this argument before. In Verification Is the New Compute Cost, the numbers showed evaluation overtaking training as the dominant cost, with a single reliable agent benchmark running into six figures. Husain’s contribution sits one layer up. Before you ask what verification costs to run at scale, ask why the output is so hard to check in the first place.

The User Inherits Your Verification Problem

Here is the line that should reorganize a roadmap: “Artifacts that are hard for you to verify are often hard for users too.” Husain’s framing is blunt, and it holds.

If your own team cannot confirm an AI-generated analysis without reconstructing it from scratch, your customer cannot either. The difficulty you feel building the eval is the same difficulty the user feels every time they open the product. A hard-to-verify output is a defect the user experiences directly, as doubt, as rework, as the quiet decision to stop trusting the feature.

Husain offers one uncomfortable anecdote to make it concrete. On one data team, roughly half the time went to reviewing half-baked AI analysis, and roughly half of that analysis turned out to be incorrect. Treat those figures as illustrative rather than measured, because that is how he presents them. The shape is what counts. When verification is expensive and the hit rate is coin-flip, the AI has not saved work. It has relocated it downstream and added interest.

Design the Output to Be Checkable

The fix is upstream of any grader: engineer verifiability into the artifact itself, so checking is fast for the builder and the user alike. Husain points at a few concrete moves.

Show where each part came from. “The fastest way to make an output checkable is to show where each part came from,” Husain writes. Provenance, source links, and citations turn a claim you have to trust into a claim you can spot-check in seconds. This is the single highest-leverage change, because it collapses verification from “reconstruct the whole thing” to “follow one link.”

Diff against a trusted baseline. An output floating in isolation forces the reviewer to judge it cold. An output presented as a delta against something already trusted, last quarter’s numbers, the approved template, the prior version, lets the reviewer focus only on what changed. You shrink the surface area that needs judgment.

Break the output into reviewable units. A single monolithic answer is verify-all-or-nothing. The same content, decomposed into discrete, individually checkable pieces, lets a reviewer approve the solid parts and interrogate the doubtful ones. Progressive disclosure serves the same goal: surface the summary, keep the evidence one click away for when trust runs out.

These are product decisions, made at design time. They sit upstream of any grader and determine whether evaluation is cheap or ruinous later.

Why Verifiability Is a Governance Concern

Verifiability designed into the product is what makes trust auditable. When an output carries its provenance, a reviewer’s approval means something specific: they checked the sources and signed off. When it does not, approval degrades into a rubber stamp, and you lose the ability to know whether anyone actually verified anything.

This is the same principle behind verification by execution: the most trustworthy check is one the system makes structurally cheap to perform. Design-for-verifiability is that idea applied at the interface. It decides, before a single metric is collected, whether your humans can supervise the AI at all or are just approving outputs they cannot actually inspect.

A product that hides its reasoning scores badly on evals and does something worse: it quietly trains its users to stop looking, and a team that has stopped looking is not governing anything.

Do This Now

Take your hardest-to-eval AI feature, the one the team keeps meaning to write more tests for, and run a different diagnosis. Sit with a real user and time how long it takes them to confirm one output is correct. If the answer is minutes, or if they cannot do it without your help, do not open the eval framework. Open the design file. Add provenance to the top claim, present the result as a diff against something the user already trusts, and split the answer into pieces they can approve one at a time. Then measure again. The eval will get easier because the product got honest.

This analysis synthesizes It’s Hard to Eval Is a Product Smell (Hamel Husain, June 2026), whose central figures are presented as the author’s own experience rather than formal study.

Victorino Group helps teams design AI products that humans can actually verify, so trust becomes measurable instead of assumed. Let’s talk.