Three Reports, One Layer: AI Risk in the Serving Stack

155 responses. That is how many Cloudflare says were affected when a KV cache bug in its serving stack exposed data across requests, a leak its security team reproduced with two independent accounts. The cause sat in the cache and the inference engine around the model.

Between October 20 and October 26, 2026, three companies published reports about their own systems. Ubicloud went first on October 20, with a measurement of tool-call accuracy under vLLM. Railway followed on October 22, with a preliminary report on a security incident. Cloudflare published on October 26, with a postmortem on cross-request KV cache exposure in its GLM-5.3 model on Workers AI. Each report was written by the company whose system failed or was measured.

Read together, they describe one layer: the software between the model and the customer. The cache that holds context. The engine settings that shape output. The store that holds the keys. Our reading is that third-party AI risk reviews need questions for each of these, and the reports themselves supply the questions.

The Inference Engine Is a Tenant Boundary

Cloudflare’s postmortem names two causes acting together. The first was a context window larger than the KV cache pool. The second was a bug in SGLang, the open source serving engine, when running in disaggregated prefill/decode mode. In Cloudflare’s words: “In total, 155 responses were affected, all to prompts longer than 681,280 tokens.” The model advertised a context window of 1,048,576 tokens, so the failure began well below the advertised limit.

The detail that should change how buyers review vendors is in the testing section. Cloudflare writes that “we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence.” A request that should have been rejected was accepted instead: “The client got HTTP 200 instead of HTTP 400.”

That sentence describes a test suite that checked the envelope and never opened the letter. Status codes were correct by the suite’s definition. The content of the response, which is the only thing the customer cares about, went unchecked at the context lengths where the bug lived.

Cloudflare’s response deserves credit. The model was taken offline within 40 minutes of declaring an incident. The company states “We have confirmed that no other model hosted on Workers AI was affected.” It also reports “We found no evidence of deliberate exploitation,” and it scopes that finding carefully: the timing and volume of affected requests were consistent with ordinary long-context use. The report also notes that the upstream SGLang bug had been reported publicly as issue #30233 on July 6, 2026.

That last fact matters for anyone running open source inference engines. The defect had a public upstream issue number. Whether a given operator tracks upstream issues against the engine version and mode it runs in production is a question a vendor review can ask directly.

We have written about tenant isolation in agent memory databases. The Cloudflare report moves the same concern one layer down. When a serving engine shares a cache across requests, the cache is where tenants meet. Isolation at that layer has to be proven on output, because a correct status code says nothing about whose data is inside the response.

Engine Defaults Are an Ungoverned Quality Change

Ubicloud’s post is smaller in stakes and just as instructive. Serving Kimi K2.6 with vLLM, Ubicloud found the model produced the expected tool-call schema “only 92% of the time.” With constrained decoding turned on, accuracy reached 100% on the newer Kimi and DeepSeek models in the K2VV ToolCall benchmark.

That 100% needs its scope attached. It comes from one vendor-run benchmark, and it covers Kimi and DeepSeek models. Ubicloud’s page reports that other models had issues with the feature enabled. The page also names specific vLLM versions for the feature and for the flag that controls it.

A later measurement fell. After an upstream vLLM default changed for requests that do not turn constrained decoding on, accuracy “dropped back to 91%.” The 92 and the 91 are separate measurements, taken at different moments, and the post reports them as such. What connects them is the mechanism: the serving engine’s configuration, set upstream, decided whether tool calls arrived in a usable shape.

For a team building agents, a tool call with the wrong schema is a broken step. At 91 or 92 percent, roughly eight or nine calls in every hundred arrive in the wrong shape. Picture, as a hypothetical, a team watching its agent’s failure rate climb: the obvious suspects are prompts, the model and its own code. The cause in Ubicloud’s account sat in an upstream engine default.

We showed a deployment flag breaking tool calls on identical weights earlier. Ubicloud’s measurement adds a second path: an upstream engine default changed, and tool-call accuracy fell on the same model without the operator setting a new flag. If your change management covers model versions and ignores engine versions, it covers half of what determines behavior.

The AI Key Is the Asset

Railway’s report is preliminary, and its language is hedged throughout. Read it with that weight. The company states that “at most 1.85% of all workspaces had some environment variables potentially compromised. This represents, numerically, about 100,000 workspaces.” Every qualifier in that sentence is doing work: at most, some, potentially.

On motive, Railway offers initial evidence only. The attacker was “likely looking for AI and platform API keys, for distilling model weights.” That is the company’s working hypothesis, stated as such, and a final report may revise it.

The commercial consequence is stated plainly: “Railway is unable to refund customers for spend performed with compromised API keys.”

Put the hypothesis and the refund policy side by side. If the hypothesis holds, the asset the attacker was after was the AI key itself, and the customer carries the cost of whatever spend the stolen key generated. Where customers kept AI keys in those environment variables, the refund policy leaves the bill with the key owner. Neither fact is about a model. Both are about where credentials live and who pays when they leak.

We have written about keeping credentials out of agent context. Railway’s incident points at the step before that: credentials sitting in a hosting platform’s environment variables, readable by anyone who reaches that store. An AI key with no spend limit, stored next to every other secret in a workspace, is a liability whose size is set by the provider’s billing ceiling.

What the Three Reports Share

In all three reports, the cause or the fix sits in the layer around the model. Cloudflare’s leak came from a cache and an engine bug. Ubicloud’s accuracy changes came from engine configuration. At Railway, the attacker reached customer environment variables after getting into its build infrastructure and CI, according to its preliminary timeline. The model sat in the middle of each story, and the risk sat around it.

The reports also share a quality worth naming. Each company published its own failure or its own measurement, with numbers. Cloudflare gave the response count and the token threshold. Ubicloud gave the percentages and the versions. Railway gave an upper bound and stated its uncertainty. That is the material a risk review should be asking every AI vendor for.

Questions about training data, bias, output safety and data retention still matter. They do not reach the cache, the engine or the key store, which is where these three reports found their problems.

Add Three Questions to Your Vendor Review This Week

Take the AI vendor questionnaire your team uses today and add these three questions. Ask them of every provider that serves models to you, including the ones you consider low risk.

How is isolation tested on output? Ask whether the vendor’s tests validate response content at the context lengths they advertise, or only status codes and latency. Ask for the longest context length at which output coherence has been verified. If the answer is the advertised maximum, ask how. Cloudflare’s report shows what the gap between those two numbers can cost.

How are engine versions pinned and verified? Ask which inference engine and version serve your traffic, how upstream changes to defaults are reviewed before rollout, and whether the vendor tracks public upstream issues against the mode it runs. Ask whether tool-call schema accuracy is measured after each engine upgrade. Ubicloud measured Kimi K2.6 schema accuracy at 92 percent, then 91 after an upstream default changed, with 100 percent reached only with constrained decoding on, in one benchmark.

Where are AI keys stored, and who pays when they leak? Ask whether your AI provider keys sit in plain environment variables on a hosting platform, what spend limits are set on each key, and what the platform’s refund policy is for spend on compromised keys. Railway has already answered the last question for its own customers.

Then turn the same three questions inward. Your own serving stack, if you run one, has a cache, an engine and a key store too.

---

This analysis synthesizes Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang (Cloudflare, October 2026), Fixing Tool-Call Accuracy in vLLM (Ubicloud, October 2026), and Preliminary Report: Oct 21st Security Incident (Railway, October 2026).

Victorino Group helps engineering organizations extend AI risk reviews from the model to the serving stack around it. Let’s talk.