The 30-Second Keepalive Is Folklore: Cache Economics Are a Per-Provider Policy

Somewhere in your agent fleet, a scheduler pings the prompt cache every 30 seconds to keep it warm. Nobody measured the interval. It came from a blog post, got copied into a runbook, and now it runs across every worker. Maxim Khailo built a harness to check the assumption, and the answer is uncomfortable: 30 seconds is roughly 8x more often than you need, and on some providers a keepalive loses money against the alternative of letting the cache die.

The measurement is public. Four providers (Anthropic, OpenAI, Google, DeepSeek), two prefix sizes, idle gaps stretched to 10 minutes, three independent runs, every call timestamped. The harness is open on GitHub with a companion arXiv paper. This is independent, self-published work, not peer-reviewed, so treat the numbers as a dated snapshot rather than a law. Provider cache behavior can change next week. That caveat is the whole point of the piece: the convention people copy has no expiry date stamped on it, and the underlying behavior does.

The eviction curves do not agree

Run the same idle-then-resume test across four providers and you get four different shapes.

At a 10-minute idle gap with a 100k-token prefix, the caches diverged hard:

Anthropic: 0 of 48 requests came back warm. There is a hard TTL cliff. After roughly 5 minutes the cache is gone, and no amount of waiting recovers it.
OpenAI: 39 of 48 warm. The cache decays gently and often survives 10 minutes.
Google: 20 of 24 warm. Gemini uses an implicit cache that tends to persist.
DeepSeek: 4 of 48 warm. Aggressive eviction, close to Anthropic’s behavior in practice.

One knob, four policies. A keepalive strategy tuned for Anthropic’s cliff is wrong for OpenAI’s slope, and actively wasteful for Gemini’s implicit cache. Copying a single interval across a heterogeneous fleet guarantees you are wrong on at least three of the four.

The 30-second convention is 8x too frequent

The folklore interval is 30 seconds. The harness tested longer ones. A 4-minute keepalive held 23 of 24 requests warm on providers with a TTL, at 7.8x less cost than pinging every 30 seconds. The warmth is nearly identical. The bill is a fraction.

The reason is mechanical. A keepalive ping is a real request against the cached prefix. You pay for it every time. Ping 8x more often than the TTL requires and you pay 8x the keepalive cost to buy the same cache residency. On a provider with a 5-minute TTL, a ping at 4 minutes and 30 seconds keeps the cache alive just as well as a ping at 30 seconds, and it costs almost nothing by comparison.

There is a formula for the break-even idle horizon. The cache is worth keeping warm only when the idle gap is short enough that a keepalive costs less than a cold recompute:

idle_breakeven ≈ τ · (w/r − 1)

where τ is the cache TTL, w is the write (cache-creation) price multiplier, and r is the read (cache-hit) price multiplier. Plug in each provider’s real numbers and the horizons scatter:

Anthropic: keepalive pays out to roughly 46 minutes of idle.
OpenAI / DeepSeek: roughly 36 minutes.
Google: roughly 12 minutes.

Every one of those horizons is far longer than 30 seconds and far longer than the interval most schedulers actually use. At a 10-minute idle gap, the 30-second convention loses money on all four providers. It buys residency you would have gotten more cheaply, or residency you did not need.

Gemini inverts the entire premise

The sharpest finding: on Google, keeping the cache warm can cost more than letting it die.

Take a 600-second idle gap and compare three strategies: let the cache expire and pay the cold recompute on resume, ping every 30 seconds, or ping every 4 minutes.

On Anthropic’s Sonnet 4.5, the cost per cycle was $0.667 to let it die, $0.867 to ping every 30 seconds, and $0.414 to ping every 4 minutes. The 4-minute keepalive saves 38% against dying. The hard TTL cliff means a resume after 10 minutes always pays full price, so a well-timed keepalive is real money saved.

On Gemini 2.5 Pro, the same three strategies cost $0.131 to let it die, $0.649 to ping every 30 seconds, and $0.186 to ping every 4 minutes. Letting the cache die is the cheapest option. The keepalive costs 40% more than doing nothing. Gemini’s implicit cache and its pricing structure mean the recompute on resume is cheap enough that any keepalive traffic is pure overhead.

Same code, opposite correct answer. A fleet policy that keeps caches warm because “warm is good” is burning money on Gemini to buy something Gemini gives away.

What this says about fleet cost governance

This is the second concrete instance of a pattern worth naming: fleet economics invert single-agent best practice. At the single-agent scale, a keepalive is a rounding error and nobody audits it. Multiply it across hundreds of workers pinging every 30 seconds around the clock, and the interval becomes a line item. The first instance was about parallelism and retries. This one is about a cache heartbeat that everyone inherited and nobody priced.

The governance point generalizes past this one knob. Conventions in this field are dated snapshots of provider behavior, and providers are actively changing the ground under them. Cache residency is becoming a metered, priced product. Anthropic already exposes cache-write and cache-read as separate line items. When residency has an explicit price, the optimal keepalive interval becomes a function you compute from current prices, not a number you inherit from a 2024 blog post. The arbitrage here has an expiry date, and so does every cost convention built on provider defaults.

This connects to how a finance function should read inference spend. An inference efficiency ratio that a CFO can govern has to account for cache economics, and cache economics are not a single number. They are a per-provider, per-model table that shifts when a provider updates pricing or TTL. A governance dashboard that assumes one keepalive policy is fine everywhere is reporting a number it did not verify.

Do this now

Pull the keepalive interval out of your agent scheduler and check it against the break-even formula for each provider you run. Three concrete moves:

Find every keepalive and read its interval. If it is 30 seconds and your idle gaps are minutes long, you are paying multiples of the necessary cost. Widen it toward the TTL minus a safety margin.
Split the policy by provider. One interval across Anthropic, OpenAI, and Gemini is wrong by construction. Compute τ · (w/r − 1) per provider from current prices and set the interval per model.
On Gemini, test letting the cache die. Measure your real resume cost against your keepalive cost. If the implicit cache and pricing make dying cheaper, turn the keepalive off for that provider and pocket the difference.

Then re-measure on a schedule, because every one of these numbers has a date on it. Rerun Khailo’s harness against your own prefixes and your own contracted prices before you trust any figure here, including his.

This analysis synthesizes Keeping the KV cache warm: measuring prompt cache eviction across Anthropic, OpenAI, and Google (Maxim Khailo, independent, with an open-source harness and companion arXiv paper, July 2026). The measurement is self-published and not peer-reviewed; provider cache behavior can change at any time, so every figure is a dated snapshot.

Victorino Group helps engineering organizations turn inference cost from folklore into a measured, per-provider policy. Let’s talk.