Your Hosting Provider Is Quietly Voting on Your AI Visibility

There is a procurement decision shaping your AI visibility right now, and it was made by your hosting vendor without asking you. It does not appear in any contract clause your team signed. It does not appear in any dashboard your team can read. It shows up only as a quiet absence: your competitors get cited by Claude and ChatGPT, and you do not.

A new investigation by Search Engine Land documents the mechanism in detail. Sites hosted on WP Engine return HTTP 429 (Too Many Requests) to several AI crawlers at the platform level. ClaudeBot, GPTBot, Amazonbot, and Bytespider all hit a wall on cache-miss requests. The block is platform-wide. It is not customer-configurable. It is not visible in the customer’s robots.txt or in any setting the marketing or web team can reach.

This is governance-as-product, executed badly. And it is happening one layer below the layer marketing teams thought they were responsible for.

The numbers buried in a status code

Searchinfluence.com, the site investigated, is hosted on WP Engine. Its citation profile across major AI assistants is asymmetric in a way that should make every CMO uncomfortable.

Google AI Mode, which has full access, cited the site at 37.8 percent. Claude, blocked at the platform layer, cited it at 0.0 percent. ChatGPT, with partial access (its newer GPTBot UA reaches the site about 54 percent of the time), came in at 9.6 percent. The correlation is not subtle. Crawler access maps directly onto citation rate. Where the bot reaches the page, the assistant cites the brand. Where the bot is rate-limited into oblivion, the brand becomes invisible.

The platform’s own support response is the artifact worth keeping. WP Engine confirmed the policy on the record as “platform-wide rate limiting on certain high-impact bots” that “cannot be selectively disabled per bot.” Translate that out of vendor language: your host has decided which AI assistants are allowed to read your content, and you are not on the approval list.

The user-agent loophole nobody planned

The investigation also found something that complicates the story in a useful way. Older Anthropic crawler user agents and Common Crawl’s CCBot still pass through. The block is keyed on specific user-agent strings, which means coverage is determined by which UA string version the AI vendor happens to send today. Anthropic upgrades its crawler ID, and your visibility drops overnight without anyone changing a setting on either side.

This is the worst possible governance posture: a policy whose enforcement depends on string matching, owned by a vendor who does not publish the list, applied to a content surface whose business value depends on being readable by the very agents being blocked. There is no audit trail. There is no change log. There is no way to know whether the policy got stricter last Tuesday or whether your traffic shifted because Claude updated its UA string.

Why this lands on marketing

Marketing teams have spent the last year being told they need to think about AEO. The advice has mostly stayed at the surface layer: structure your content for agents, expose clean metadata, write in passages an LLM can quote. We argued the operational version of that case in The Agent-Readable Surface Is Now a Marketing Deliverable.

The WP Engine pattern means the surface work is necessary but not sufficient. A perfectly structured, agent-ready page on a host that returns 429 to the agent is invisible. The procurement decision two layers below the CMS, made by an infrastructure vendor, overrides every editorial choice the marketing team makes.

This is the same shape as the budget-burn problem we covered in Marketing’s MCP Moment. Vendors are shipping policy decisions inside platform features without telling the buyer. The buyer signed a hosting contract. The vendor delivered an AI-visibility policy. Those are different products. The buyer is paying for both and only sees the invoice for one.

A diagnostic you can run today

The good news is that the test is fast and cheap. You can confirm or rule out the block on your own site in under five minutes.

curl -A "Mozilla/5.0 (compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" \
  -I https://yoursite.com/

A 200 response means you are reachable. A 429 means your host is filtering at the platform layer. Run the same test with GPTBot/1.2, Amazonbot/0.1, and Bytespider. Document the responses. The output is the audit log your hosting vendor is not giving you.

If you get 429s, the next move is governance, not engineering. Open a ticket with platform engineering at the host. Frame the conversation as an exceptional use case for your specific account, not a generic policy debate. Some platforms will whitelist on request. WP Engine support has not, in the cases on record. If the answer is no, the choice is between accepting zero AI visibility on the affected assistants or migrating. Kinsta, Pressable, and Pantheon do not run this block today. The migration is a real cost. Zero citations from Claude is also a real cost.

The procurement implication for next quarter

The hosting vendor is now an AEO buyer on your behalf. So is the CDN. So is the WAF. So is whatever bot-management product was bundled into your enterprise security stack last year. Each of those layers has a default policy on AI crawler traffic. Most of those defaults were set before AEO became a strategic concern. The defaults are now shaping your visibility, and almost none of them are documented in a place the marketing team can see.

The procurement workstream for next quarter looks like this:

Inventory every layer between your origin and the public internet. Host, CDN, WAF, bot-management. Name the vendor for each.
For each layer, request the current AI crawler policy in writing. ClaudeBot, GPTBot, PerplexityBot, Amazonbot, Bytespider, CCBot. Yes or no on each.
Get the change-notification clause in writing too. If the vendor updates the block list, when do you find out? Today, the answer is usually never.
Run the curl diagnostic monthly. Treat the response codes as an SLI. A change in the response is an incident, the same way a change in uptime is an incident.
Negotiate the policy explicitly at contract renewal. Hosting contracts now need an AI-crawler addendum. The vendors will not write it for you.

This is the procurement equivalent of the governance work we covered in Governance Ships as Product. The platform is making the policy. The buyer’s job is to make the policy visible, contestable, and contractually defined.

What changed this week

The investigation that surfaced this is from May 2026. The behavior has been live for longer. The reason it matters now is that AI-assistant citations have crossed the threshold from interesting metric to revenue signal. When 38 percent of your category visibility on Google AI Mode is being cited and 0 percent on Claude is being cited, the asymmetry is no longer a curiosity. It is a structural disadvantage your competitors with different hosting choices are not paying for.

The fix order is unromantic. Test. Document. Escalate. Migrate if necessary. None of these are creative work. All of them are the kind of operational hygiene that distinguishes a marketing team that owns its AI visibility from one that ships content into a black box.

The vendor cast a vote on your AI strategy. Read the ballot.

This analysis synthesizes Your Managed WordPress Might Be Blocking AI Bots and You Can’t See It (Search Engine Land, May 2026).

Victorino Group audits the hosting and infrastructure layer that quietly shapes AI visibility for marketing teams. Let’s talk.