Rigging the Search Layer Buys Mentions, and Picks Only in Part
Joe Shirey ran 5,580 trials to see whether you can optimize your way into an AI agent’s recommendation. The most aggressive arm pushed the agent to search (925 of 930 trials did) and rewrote every query to favor Google Cloud. Compared with the search-encouraged arm without the rewrite, mentions rose from 68.3% to 89.1%. Selections as the primary recommendation rose from 13.0% to 17.1%. With no search at all, the agent selected Google Cloud in 9.2% of trials.
Those numbers carry two lessons. Manipulating what an agent reads moved mentions by 20.9 points and selections by 4.1, from different starting levels (68.3% and 13.0%). And before any manipulation, one variable matters: whether the agent searches at all.
What was tested
Shirey is a Google Cloud employee and says so on the page. He used 310 prompts, each run 3 times, for 930 trials per arm and 5,580 in total across six arms. The agent was Claude Code running Claude Sonnet 5, with its native search blocked and replaced by an MCP search tool. He reports using Gemini-grounded search.
The baseline reflects how the prompts were built, since every prompt had a Google Cloud competitor. It says nothing about market share. The study covers one model and one vendor, answers advisory architecture questions, and reports no confidence intervals. Read the figures as one experiment, not a measurement of the market.
The table
Shirey’s results, for “selected as the primary recommendation” and “mentioned”:
| Arm | Selected | Mentioned |
|---|---|---|
| Baseline | 9.8% (91) | 57.6% |
| No search | 9.2% (86) | 56.0% |
| Search available | 10.0% (93) | 59.7% |
| Search encouraged | 13.0% (121) | 68.3% |
| Hacked and available | 11.0% (102) | 63.5% |
| Hacked and encouraged | 17.1% (159) | 89.1% |
Compare the two encouraged arms. Shirey writes that the bias “added 20.9 percentage points to mentions, but only 4.1 points to actual selections.” That comparison runs from search-encouraged to hacked-and-encouraged. Rewriting the queries moved what the agent said about the vendor much more than what it chose.
The “hacked and available” row moved mentions from 59.7% to 63.5% and selections from 10.0% to 11.0%.
Search is a switch someone holds
For governance, the figure we weigh most is a count of searches. Shirey reports that “Sonnet 5 searched in 3 out of 930 trials, or 0.3%” at baseline. In one of the two encouraged arms, the agent searched in 925 of 930 trials (99.5%).
At baseline this model almost never searched. In an encouraged arm it nearly always did. In our reading, whether the retrieval layer enters the decision is a matter of configuration. How the encouragement was implemented is not in our briefing.
We argue that the control surface sits with the operator: the system prompt and the tool list are where search availability and encouragement would be set. Where search is only available, selection barely moved (10.0%, against 9.2% with no search). Where it was encouraged, selection reached 13.0%.
One consequence for buyers: an agent that advises on vendor choice may behave differently depending on its configuration, which a reader of its answers may not see.
The rewrite the agent never saw
The hacked tool rewrote every query the agent sent. Shirey’s observation: “the agent never noticed that every query it sent had been rewritten. If you plug a third-party search tool into your agent, you’re trusting it with more than you might think.”
We would add a framing of our own. A search tool that can rewrite queries is a component that decides what the agent gets to read. Nothing in this experiment tells us whether any real tool rewrites queries. In this experiment the agent did not notice the change, and nothing in the study shows it could see the rewritten query.
That makes a third-party search tool an unaudited trust surface, in the same class as any other dependency whose behavior changes what your agent concludes. The study gives the review a concrete shape: compare what the agent asked for with what the provider received.
What a second dataset suggests, with limits
Grow and Convert publishes one data set, and it is a vendor that sells content and SEO services. For one unnamed B2B client, 7,779 answers mentioned the client, and in 87% of those the client’s own site appeared in the sources. In AI Overviews, the client was mentioned in 90% of answers when its own site was cited, against 25% when only a third-party page about it was cited.
Handle this with care. It is one client. It is correlational. The 87% is conditional on a mention, so it does not mean 87% of all answers cite the client’s site. The denominator behind the totals is unclear on the page. We use it only to note that answer engines that cite a brand’s own site name it more often, a pairing on the retrieval side that Shirey’s rewrite pushes with a very different method.
What a vendor can and cannot buy
Putting the two sources side by side, the evidence supports a modest claim. In this test, getting the retrieval layer to favor Google Cloud lifted mentions by 20.9 points and selections by 4.1 points, from different starting levels (68.3% and 13.0%). In relative terms both rose by roughly 30%.
Anyone reporting AI visibility as a win should ask which of the two was measured, the mention or the pick.
For the people running agents, our concern points the other way. A mechanism that lets a test tilt mentions could in principle be operated by a tool provider or an intermediary, and the agent would not report it.
For related reading, see AEO Is Four Separate Games, Not One, Your AI-Search Citations Expire Every 11 Days and 42% Agreement.
Do this now
Inventory every search or retrieval tool your agents can call, and for each one write down four facts. Who configures whether it is used. Whether the system prompt tells the agent to use it. What the tool receives from the agent, and what it sends onward to its provider. Whether you log both sides of that hop.
Then ask each third-party tool vendor in writing whether it modifies, expands or re-ranks queries, and keep the answer. If an agent advises on purchases, vendors or architecture, log how often it searches, because in our reading the search rate sets how much of its answer your retrieval layer can influence.
This analysis synthesizes Can You SEO Your Way Into an AI Agent’s Recommendation? (Joe Shirey’s Blog, October 2026) and Is 3rd party data the biggest lever in GEO? Our data says no. (Grow and Convert, September 2026).
Victorino Group helps teams audit the search and retrieval tools behind their agents and set who controls them. Let’s talk.
All articles on The Thinking Wire are written with the assistance of Anthropic's Opus LLM. Each piece goes through multi-agent research to verify facts and surface contradictions, followed by human review and approval before publication. If you find any inaccurate information or wish to contact our editorial team, please reach out at editorial@victorinollc.com . About The Thinking Wire →
If this resonates, let's talk
We help companies implement AI without losing control.
Schedule a Conversation