Skip to main content

Prompt Caching Across Providers

Advanced

Prompt caching is the single largest cost lever in a production LLM app — routinely a 90%+ cut on the input side. Every major provider now has it. That makes it easy to assume it's the same feature everywhere, and it isn't.

The mechanism is identical: the provider reuses the processed prefix of your prompt instead of re-reading it. What differs is who decides, how long the cache lives, how long a prefix has to be to qualify, and — the one that bites — whether you pay a premium to write it. A workload tuned on one provider can quietly stop caching, or cost more than not caching at all, on the next.

What you'll learn
  • Tell apart the two control models — explicit breakpoints vs automatic prefix detection — and what each costs you
  • Read the cross-provider table: minimum prefix, TTL, read discount, write premium, and the usage field that proves a hit
  • Apply the portable break-even rule: how many reuses inside the TTL before caching pays
  • Avoid the five traps that silently kill a cache when you port a workload between providers
  • Verify a real hit rate on any provider from the response usage, not from the docs

If you only work on Claude, read Prompt Caching & Cost Optimization first — it teaches the cache_control mechanism in depth. This page is the portability layer on top of it.

The two control models​

Everything in this space is one of two designs, and the design decides your failure mode.

Explicit (Anthropic): you attach a marker to the last stable block. Nothing is cached until you ask. You get exact control over the boundary and can place several. You also carry the bug: a prompt with no breakpoint pays full price forever, and nothing warns you.

Automatic (OpenAI, Google's implicit caching, DeepSeek): the provider hashes your prefix and reuses what it can. Nothing to configure, and nothing to fix when it misses. Your only lever is prompt layout — put stable content first — plus, on OpenAI, a routing hint.

The lever you keep in both models

Prefix order is the one control that works everywhere. Stable content first, volatile content last. On explicit systems it decides where your breakpoint can go; on automatic systems it's the only thing you control.

The cross-provider table​

Anthropic (Claude)OpenAIGoogle (Gemini)DeepSeek
ControlExplicit cache_control, up to 4 breakpointsAutomatic, prefix-matchedImplicit by default; explicit cache objects also availableAutomatic, on disk
Minimum prefix512 tokens on Fable 5.1 / Mythos 5.1 / Opus 5.5 / Opus 5 / Fable 5 / Mythos 5; 1,024 on Sonnet 5 and Opus 4.8; 2,048–4,096 on older models1,024 tokens on GPT-5.6 and later; varies by request settings before that4,096 tokens on Gemini 3.5–3.8 Flash and 3.1 Pro Preview; 2,048 on Gemini 2.5 Flash / ProNot published as a token count — cache units land at request boundaries, detected common prefixes, and fixed token intervals
TTL5 minutes default; 1 hour optional30 minutes default on GPT-5.6+ via prompt_cache_options.ttl; earlier models use prompt_cache_retention — in-memory (typically 5–10 min) or 24hNot a fixed published default for implicit cachingAutomatic eviction "within a few hours to a few days" once unused
Cache read0.1× base input; 0.05× on Opus 5.5; 0.025× on Fable 5.1 / Mythos 5.10.1× base inputPublished as a separate cached-input rate (0.1× the standard rate on Gemini 3.8 Flash through 2026-12-31)Separate cache-hit input rate, roughly 1/30th of the miss rate on deepseek-v4-pro
Cache write premium1.25× base input for the 5-minute tier; 2× for the 1-hour tier1.25× base input on GPT-5.6+None for implicit caching; explicit caching bills storage at $0.50 per million tokens per hour on Gemini 3.8 Flash through 2026-12-31None published
Proof of a hitcache_read_input_tokens and cache_creation_input_tokens in usageCached-token count in usageusage.total_cached_tokensprompt_cache_hit_tokens and prompt_cache_miss_tokens
Accounting / routing hintBreakpoint placementprompt_cache_key — separate cache accounting on GPT-5.6+, cache routing before that——

For self-hosted open models the same idea exists one layer down as prefix caching in the serving engine: vLLM and SGLang reuse KV-cache blocks for a shared prefix. There's no per-token price, so there's no break-even to compute — the cost is VRAM, and the win is prefill latency, not the bill. vLLM exposes it as enable_prefix_caching; on the V1 engine it is on by default, so on a current build you're more likely to be turning it off than on. See Run models locally with Ollama and A private local AI stack.

The break-even rule​

Here is the part the provider docs leave as an exercise. Caching is not free on explicit systems, and since GPT-5.6 it isn't free on OpenAI either. A write premium means a prefix you reuse only once can cost you more than not caching it.

Let P be what the prefix would cost at the base input rate, w the write multiplier, r the read multiplier, and N the number of calls that share the prefix inside one TTL window.

Without caching:  N · P
With caching:     w · P  +  (N − 1) · r · P

Caching wins when:   w + (N − 1) · r  <  N
Which solves to:     N  >  (w − r) / (1 − r)

Plug in the published multipliers:

TierwrBreak-even NReuses needed
Anthropic 5-minute, Opus 5.51.250.051.262 calls
Anthropic 5-minute, standard read tier1.250.11.282 calls
Anthropic 1-hour20.12.113 calls
OpenAI GPT-5.6+1.250.11.282 calls
Automatic, no write premium10.101 call — never a loss

So the rule of thumb is short: two hits inside the TTL and you're ahead; make it three before you buy the 1-hour tier. Below that, caching a prefix is a small, silent loss.

Pro tip
  • N counts calls inside one TTL window, not calls per day. A prefix reused 1,000 times a day in bursts 20 minutes apart re-pays the write on every burst.
  • The 1-hour tier is not a straight upgrade. It doubles the write premium to buy a window 12x longer — worth it only when your traffic is genuinely sparse.

Worked example: a 50k-token prefix, 1,000 calls a day​

On Claude Opus 5.5 — base input $4/MTok, 5-minute write $5/MTok, read $0.20/MTok, all from Anthropic's pricing table:

Guided walkthrough1 of 4
  1. 50,000 tokens x 1,000 calls = 50M input tokens. At $4/MTok that is $200/day, every day, for content that never changed.

The teaching point is step 3 versus step 2: the same call volume, the same prefix, and a 3.3x difference in bill, decided entirely by arrival pattern against the TTL. Arrival pattern is the variable nobody puts in the spreadsheet.

Five traps when you port a workload​

Watch out
  • TTL gap: a job that runs every 10 minutes hits on OpenAI's 30-minute default and misses on Anthropic's 5-minute default. Same code, same prompt, zero hit rate.
  • Threshold gap: a 2,500-token system prompt caches on Claude Opus 5.5 (512-token minimum) and on GPT-5.6 (1,024) but not on Gemini 3.8 Flash (4,096). It fails silently — no error, just a full-price bill.
  • Invalidator gap: every provider invalidates on a changed prefix, but the non-obvious triggers differ. On Anthropic, changing tool definitions, tool_choice, or (model-dependent) thinking and effort settings invalidates. On OpenAI, changing the model, tool definitions, schemas, reasoning effort, verbosity, or context compaction does.
  • Write-premium gap: porting a low-reuse prefix from an automatic provider to an explicit one can make it more expensive than no caching. Run the break-even rule above before you mark a breakpoint.
  • Observability gap: the usage field names share no convention. Any dashboard that parses one provider's cache fields needs a per-provider adapter, or your hit rate reads as zero after a migration.

The one Anthropic-specific mechanic with no analogue elsewhere: the cache is looked up over a 20-block lookback window per breakpoint. A steadily growing conversation can outrun that window and stop hitting even though the prefix is unchanged — see Prompt Caching Economics for the invalidation rules in full.

Verify it, don't assume it​

Docs describe intent; usage describes what you were billed. Check the real hit rate on every provider you run on.

Audit a workload's cache hit rate

Audit prompt caching for this codebase.

1. List every place we call an LLM API, with the provider and model.
2. For each call site, identify the prefix that is stable across calls
 and anything volatile that appears BEFORE it (timestamps, user IDs,
 reordered tool lists, freshly serialized JSON with unstable key order).
3. Report the token length of each stable prefix and compare it to that
 provider's minimum cacheable length. Flag any prefix below it.
4. Estimate the typical gap between consecutive calls sharing each prefix
 and compare it to that provider's cache TTL. Flag any gap that exceeds it.
5. Show me where we read the cache fields back from the response usage.
 If we never read them, say so plainly — that is the finding.

Do not change any code. Give me a table of call site, provider, prefix
tokens, minimum, typical gap, TTL, and the fields we read.

Two failure signatures worth naming, because they look identical on a dashboard and have opposite fixes:

  • Cached tokens always zero on repeated requests that should share a prefix → a volatile value sits before your stable content, or the prefix is under the minimum. Diff the rendered prompt bytes between two calls.
  • Cached tokens nonzero but the bill barely moved → the prefix is caching but it's short relative to your volatile suffix. Caching is working; it isn't where your money is going. Look at output tokens and why agents burn tokens instead.
Lock in the terms
Press Enter or Space to flip the card. Use the left and right arrow keys to move between cards.Term shown.
1 / 6

Check yourself

0/4
  1. A prefix will be reused exactly twice inside the window. Which tier is the wrong choice?
  2. A 2,500-token system prompt cached fine on Claude and stopped caching after a port to Gemini 3.8 Flash. Most likely cause?
  3. Same prompt, same 1,000 calls a day. Why would the bill triple?
  4. Cached tokens read nonzero, but the bill barely moved. What does that tell you?
Key takeaways
  • Two designs, two failure modes: explicit caching fails by never being marked; automatic caching fails silently and you cannot force a hit.
  • Caching is not automatically free. With a write premium you need N > (w - r) / (1 - r) reuses inside one TTL — two calls for the 5-minute tiers, three for Anthropic's 1-hour tier.
  • Arrival pattern against TTL, not call volume, decides the bill: the same 1,000 daily calls cost about $10 evenly spread and about $34 in bursts on Opus 5.5.
  • Minimum prefix length spans 512 to 4,096 tokens across providers — a prompt that caches on Claude can silently stop caching on Gemini Flash.
  • Prove the hit rate from the response usage on every provider; the field names share no convention, so a dashboard needs a per-provider adapter.
  • Stable content first, volatile content last. It is the one lever that works on every provider, including self-hosted prefix caching.

Sources & further reading​

Next​