One Toggle Flips 80% of ChatGPT's Product Recommendations

Flip one setting and 80.2% of ChatGPT’s product recommendations change. That is the headline number from a study of 20,000 responses by Visibility Labs, which ran 1,000 “what is the best ___” prompts ten times with search enabled and ten times with it off. The two sets of answers barely resembled each other.

Only 19.8% of recommendations overlapped between search-on and search-off. The same prompt, the same model, one toggle, and four out of five product suggestions came back different.

The Number That Should Reset Your Mental Model

Most marketing teams still treat AI recommendations as a fixed property of the model. They imagine ChatGPT “knows” the best CRM the way it knows the capital of France, and they try to teach it by seeding brand mentions across the web. The study breaks that assumption.

Of the products recommended 100% of the time with search off, only 15.8% kept that perfect score once search was on. A brand that was a guaranteed recommendation from training data became a coin flip the moment the model started reading the live web. The training-data answer and the search answer are two different commercial worlds.

The response shape shifted too. With search on, ChatGPT recommended 5.2 products per response and surfaced 19 unique products per prompt. With search off, those numbers rose to 6.2 and 21.8. Search narrows the field and reorders it.

Where Recommendations Actually Come From

Per the study, the lever is the cited-source layer. A companion analysis of 10,000 responses measured a 0.4 Pearson correlation between how often a brand appeared in ChatGPT’s cited sources and how often it got recommended. At 10% citation visibility, a brand averaged 0.4 mentions in the recommendation. At 100% citation visibility, it averaged 3.0. The relationship is direct and roughly linear.

Visibility Labs states the implication plainly: “Getting mentioned in the cited sources is one of the most impactful things you can do to get recommended more often in ChatGPT.” And on the old playbook: “Trying to influence the training data by scattering brand mentions across the web is a bit of a lost cause.”

This reframes the entire optimization target. When search is on, and increasingly it is the default, the model’s recommendation is assembled at query time from the handful of pages it just retrieved and cited. Those pages are observable. You can see them. That changes what governance and measurement can actually do.

Non-Determinism Is Now a Commercial Property

Hold the data points together and a harder fact emerges. The same buying-intent prompt produces roughly 80% different commercial answers depending on one configuration setting the user never sees and you never control.

This is non-determinism as a feature of AI-mediated commerce, not a bug to be patched. A customer asking “what is the best project management tool” might get your brand or might not, and the deciding factor is whether their ChatGPT session has search enabled and what the model retrieved in that instant. You cannot make the answer deterministic. You can only influence the inputs the model reads when search is on.

Two consequences follow for any team that sells through AI answers.

First, a single recommendation tells you almost nothing. If 80% of answers swing on one toggle, then one screenshot of ChatGPT recommending you, or recommending a competitor, is noise. Measurement has to run the same prompt many times, across configurations, and report distributions. A brand that shows up in 70% of search-on responses is in a genuinely different position than one that shows up in 20%, and you cannot tell them apart from a single check.

Second, the auditable surface is the citation list, not the model’s memory. You cannot inspect or edit what the model absorbed in training. You can inspect which sources it cites for a given query today, whether your domain is among them, and what those sources say about your category. The cited-source layer is the place where measurement is possible and where influence is legitimate.

The Governance Surface Just Moved

For two years the governance conversation about AI visibility has pointed at the wrong layer. Teams asked how to get into the training data, how to rank in the retrieval index, how to reverse-engineer the model’s preferences. The study relocates the problem to something concrete: the sources ChatGPT cites at answer time, and your presence within them.

That is governable in a way the model’s internals never were. A citation either includes your domain or it does not. A cited page either represents your product accurately or it misrepresents it. A category query either pulls sources where you are visible or sources where you are absent. Each of these is observable, trackable over time, and assignable to an owner.

It also means the failure mode is auditable. If a competitor consistently wins the search-on recommendation, you can pull the cited sources and see why: they are present in the pages ChatGPT retrieves and you are not. That is a finding you can act on, unlike “the model prefers them,” which is a dead end.

The 0.4 correlation is not destiny. It says citation presence strongly drives recommendation frequency, while leaving room for ranking, recency, and how the sources describe you. The teams that win will treat citation presence as a measurable KPI, instrument it, and manage it like any other governed surface.

Do This Now

Pick your ten highest-intent category prompts, the “what is the best ___” questions a buyer would actually ask. Run each one in ChatGPT at least ten times with search enabled, and record two things: whether your brand appears in the recommendation, and which sources the model cites. Then run them with search off to see your training-data baseline.

You now have a distribution instead of an anecdote, and a list of the exact sources deciding your visibility. Those cited domains are your work queue. Get accurate, present, current coverage into them, and re-measure. That loop is the governance surface for AI-mediated commerce, and per this data, it is roughly 80% of the game.

This analysis synthesizes ChatGPT Search vs No-Search Product Recommendations (Visibility Labs, June 2026).

Victorino Group helps marketing teams govern and measure their visibility inside AI answers. Let’s talk.