Skip to main content

Prompt Caching & Cost Optimization

Advanced

If many of your requests share a large, unchanging chunk — a long system prompt, a big document, a tool catalog — prompt caching lets the API reuse the processed prefix instead of re-reading it every call. That cuts both cost and latency on the cached part.

What you'll learn
  • The mental model: a cache breakpoint after a stable prefix, reused across calls
  • How to mark the breakpoint in Python and TypeScript with cache_control
  • The one invariant that makes or breaks it — the prefix must be byte-for-byte identical
  • How to read usage fields to confirm you are actually getting cache hits
  • Where caching pays off most, and how to pair it with batching and right-sizing

How it works (the mental model)​

You mark a cache breakpoint after the stable prefix. On the first call it's processed and cached; subsequent calls that share the exact same prefix hit the cache and pay much less for it.

Caching vocabulary
Press Enter or Space to flip the card. Use the left and right arrow keys to move between cards.Term shown.
1 / 4

Mark the breakpoint (copy-paste)​

Add cache_control to the last stable block — here, a large system prompt. The user's turn comes after it and varies freely; everything up to and including the marked block is cached.

Guided walkthrough1 of 4
  1. Find the large, unchanging chunk — a long system prompt, a big document, or a tool catalog reused across many requests.
import anthropic

client = anthropic.Anthropic()

message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LARGE_STABLE_PROMPT,  # long, unchanging — the cached prefix
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": "Summarize the key points."}],  # varies per call
)

print(message.usage.cache_read_input_tokens)  # > 0 means you got a hit

The first call pays a small write premium to populate the cache; every later call with the same prefix reads it back at a fraction of the input price. The prefix must be long enough to be eligible — a few thousand tokens, model-dependent — or it silently won't cache.

The invariant that makes or breaks it​

Caching is prefix-exact

A cache hit requires the cached prefix to be byte-for-byte identical. The most common bug: a silent invalidator near the top of the prompt — a timestamp, a changing user name, a reordered tool list — that changes the prefix and quietly drops your hit rate to zero.

Put everything stable first, everything variable last, and keep the prefix truly constant.

Check that it's actually working​

Don't assume — read it back from the response usage:

  • cache_creation_input_tokens — tokens written to the cache this call (the first request).
  • cache_read_input_tokens — tokens served from the cache (the savings).
  • input_tokens — the uncached remainder, billed at full price.

If cache_read_input_tokens stays zero across repeated requests that should share a prefix, either a silent invalidator is at work or the cache entry simply expired. You no longer have to diff prompt bytes to tell which: the API will name the divergence point for you. See Cache Diagnostics.

Where it pays off most​

  • Long system prompts reused across users.
  • RAG / document Q&A where the same source text is queried repeatedly.
  • Agents with a fixed tool catalog and instructions over many turns.

Pair caching with batching for offline workloads — the discounts stack, and the 1-hour cache duration is designed for exactly that combination — and with right-sizing the model (Choosing a Model) for the biggest combined savings — see Cost & Latency.

Check yourself

0/3
  1. What does a cache hit require of the cached prefix?
  2. Which usage field tells you tokens were served from the cache (your savings)?
  3. Where should the variable, per-call content go relative to the cache breakpoint?
Key takeaways
  • Mark a cache breakpoint after the stable prefix; the first call writes it, later calls read it back cheaply.
  • A cache hit needs a byte-for-byte identical prefix — keep stable content first, variable content last.
  • Silent invalidators near the top of the prompt (timestamps, names, reordered tools) quietly drop hit rate to zero.
  • Verify with usage: cache_read_input_tokens > 0 means a hit; zero across repeated requests means an invalidator is at work.
  • Caching pays off most for reused system prompts, RAG, and agents; combine it with batching and model right-sizing.

Next​