An Approval Gate on the Wrong Objective Just Approves Bad Changes Faster

LangChain published the build notes for a paid media agent that holds write access to ad platforms. The control model around that write access is the most complete one we have seen published: only designated team members can approve, verified by user ID, with everyone else read-only. Every recommendation arrives in Slack as a card showing the current value against the proposed value. After the change is applied, code verifies that it actually landed on the platform. Capability profiles differ per entry point, so a scheduled report cannot do what a Slack conversation can. Each request runs in an isolated sandbox with its own checkpoint, and subagents are limited to reading context, computing and rendering, writing to separate report locations.

That is a serious piece of engineering. It is also aimed one layer below the decision that matters.

What the approver is actually approving

The seven steps are explicit: analyse platform and warehouse data, generate recommendations with supporting evidence, post the approval card, wait for an authorized reviewer to approve or edit, apply the change through code, verify it landed, report back. The source-of-truth assignment is equally explicit. Ad platforms own media metrics such as spend, impressions and clicks. The warehouse owns downstream outcomes such as leads, opportunities and pipeline. Platform-specific limits and attribution models are disclosed in the output rather than hidden.

That boundary is drawn in the right place. Spend, impressions and clicks are counts the platform observes directly. Leads, opportunities and pipeline are counts your warehouse observes directly. The connective tissue between them, the part that says this spend produced that pipeline, is neither. It is an attribution model, and the agent’s recommendations are shaped by it.

The measurement underneath

Gordon, Zettelmeyer, Bhargava and Chapsky published in Marketing Science in 2019 the comparison that settles the epistemic status of that connective tissue. Fifteen large-scale U.S. advertising experiments. Five hundred million user-experiment observations. Roughly 1.6 billion ad impressions. They ran randomized experiments alongside the observational methods practitioners use to estimate the same effect: exact matching, propensity score matching, regression adjustment.

The observational methods were, in the authors’ framing, frequently overestimating incremental conversions by a factor of three or more. They were also sometimes underestimating. That is not an average error and it is not a ceiling. It is a distribution whose direction you cannot know in advance for your own campaign. Gordon, Moakler and Zettelmeyer replicated the work across hundreds of experiments in Marketing Science in 2023 and reached the same conclusion at larger scale. A 2017 paper in the Journal of Marketing Research proposed the ghost ads approach as one way to get a valid counterfactual more cheaply.

The mechanism is unglamorous. Ad delivery systems target people who were already more likely to convert, and no adjustment on observables fully removes selection on unobservables. The measurement problem is structural, not a matter of better covariates.

Two things follow. The evidence is from 2019 and 2023, so nobody can claim this is a fresh discovery that governance has not had time to absorb. And platforms ship two different classes of number through the same dashboard: genuine randomized experiments, and observational or modeled attribution. The dashboard does not label which is which. As Karan, who writes about running measurement for a $1B portfolio, puts it, the ambiguity serves the vendor.

The join worth making

We have written before about the attribution loop you cannot audit and about autonomous marketing agents turning brand voice into a data-governance problem. This is the case where both meet a third thing: the agent now holds spend authority, and the approval gate around that authority is good.

A good gate on an invalid objective produces a specific failure. The approver sees current spend, proposed spend, and the evidence the agent assembled. The evidence is internally consistent. The diff is accurate. The verification confirms the platform accepted the change. Every control fires correctly. And the allocation still moves toward whichever channel the attribution model happens to overstate, because nothing in the workflow asks whether the lift number was produced by randomization or by adjustment on observables.

No claim is being made that LangChain’s agent caused a bad outcome. There is no evidence of that, and their self-reported numbers point the other way. The point is narrower and harder: this class of error is invisible to the control model as described. A human approving a well-formatted diff against a modeled number approves it faster than a human building a spreadsheet, which is the entire value proposition.

Four controls the described model does not include

Reading the LangChain post against what a spend-authority control model needs, four things are absent from the description:

No stated spend ceiling or blast radius cap per approved change. An approval is an approval, whether it moves $200 or $20,000.
No rollback procedure. Verification confirms the change landed. Nothing described reverses it.
No segregation of duties. The same agent proposes the change, applies it and verifies it. The human sits in the middle of that chain, not at either end of it.
No audit trail outside the vendor’s own stack. The record of what was proposed, approved and applied lives in the system that produced it.

Those four are what a finance team would demand before granting a junior analyst the same authority. The same asymmetry shows up whenever an agent gets write access to a system of record, and advertising carries its own governance surface on top. The gap between an agent’s permissions and the controls a person with identical permissions would face is where governance debt accumulates.

On the vendor’s numbers

LangChain reports pipeline attribution moving from 0 to 20% in six months, cost per qualified lead down about 30% from June to August, monthly spend up about 60%, LinkedIn CPL 40% lower than January, about $5K a month saved versus an agency, a reporting workflow about 40x cheaper and 13x faster, runtime from 1,112 seconds to 85 seconds, and early input consumption of about 3.9 million tokens per report reduced to an optimized first-turn context of about 12,000 tokens.

Every one of those is self-reported by a vendor about its own marketing, produced by its own agent running on its own commercial stack, with no external audit, no published methodology and no denominators disclosed. Treat the engineering claims (token counts, runtime, the context architecture) as more credible than the marketing outcomes, because the first kind is measured inside a system they control and the second kind depends on exactly the attribution layer this article is about.

Their architectural claims are worth keeping regardless. Five context layers ordered by how often each changes: system prompt, skills, wiki, live tools, deterministic code. The test for what belongs where is clean enough to steal: a skill should work at another company, whereas the wiki should not. Python fetches data, aligns date windows, calculates totals and comparisons, applies fixed rules and writes compact results to the sandbox, leaving the model only what requires judgment. And the line that most teams need: the context window is often the bottleneck, not the model.

Do this now

Take the three intake questions Karan proposes and apply them to every number your agent is allowed to optimize, before you grant it write access to anything.

Was assignment randomized, and by what mechanism? A yes or no, with a documented mechanism. Who holds the holdout, and can it be audited? Third-party measurement, exportable results, a design you co-specified. Does the outcome metric reconcile to anything recognized? It should walk to financial quantities and pipe into your own measurement stack.

Then adopt the posture that follows from those answers. Label every platform number experimental or observational at the moment it arrives. Experimental results with randomization, adequate power and reconcilable outcomes are evidence, and may calibrate your models. Everything else is operational telemetry, and may not drive allocation.

Karan locates the failure precisely: organizations allocating on modeled lift aren’t uninformed, they’re unwilling. An agent with spend authority removes the last excuse, because the thing that used to make this tolerable was that a human moved budget slowly enough for the error to wash out. Approve at machine speed against a number you never classified, and it does not.

This analysis synthesizes Facebook proved its own ad measurement wrong (Karan, karandhir.substack.com, September 2026) and How we built LangChain’s paid media agent (LangChain, A. Irgashev, D. Lambert and J. Gomez, September 2026).

Victorino Group helps teams classify the metrics their agents optimize before granting those agents authority to act on them. Let’s talk.