إنتقل إلى المحتوى الرئيسي

Mid-Conversation Tool Changes

متقدّم

For as long as Claude has had tool use, the tools array has been frozen for the life of a conversation — or, more precisely, frozen for the life of a cache entry. Change it and the prompt cache melts.

That is because prompt caching hashes the request prefix in a fixed order: tools → system → messages. The tool list sits earlier than anything else you send. Add one tool, rename one description, and every cached turn after that point misses. On a long agentic session with hundreds of thousands of cached input tokens, that "small edit" can cost you real money and a fresh multi-second cold start.

Mid-conversation tool changes are the tool-array counterpart to mid-conversation system messages. You still declare the full universe of tools in tools once, up front. But you now decide which subset is actually offered to the model on any given turn by appending tool_addition and tool_removal blocks inside a role: "system" message. The tools array itself never changes, so the cached prefix stays byte-identical.

What you'll learn
  • Why editing tools[] used to blow up the whole cache, not just the tool section
  • How defer_loading, tool_addition and tool_removal split declaration from availability
  • The exact placement rules for the system message that carries these blocks (they inherit the rules from mid-conversation system messages)
  • How to reference MCP tools individually (mcp_tool_reference) or as a whole server (mcp_toolset_reference)
  • When this beta beats the alternatives — sub-agents with their own tool_choice, per-turn resend, or an outer router

★ Insight ───────────────────────────────────── Two things make this feature quietly important. First, on Opus 5 the minimum cacheable prompt dropped from 1,024 to 512 tokens, so smaller sessions benefit from the cache — which means small sessions now also suffer when you invalidate it. Second, tools sitting before system in the hash means today, when you use mid-conversation-system-messages to sneak in a new instruction, you still pay full price the day you need to introduce a new tool. This beta closes the last hole. ─────────────────────────────────────────────────

The cache-hash problem in one picture​

A request's cache key is a rolling hash of the prefix, in this order:

[ tools ][ system ][ messages…, up to the breakpoint ]

A cache hit requires every byte before the breakpoint to match a recent request. So:

What you changeWhat still hits cacheWhat you re-pay
Append a new user turn at the endThe whole prefix up to that turnOnly the new turn
Append a new mid-conversation system messageEverything before itThe new system message
Edit the top-level system fieldOnly toolssystem + every message
Add one new tool to toolsNothingsystem + every message

That last row is the one Mid-Conversation Tool Changes rewrites.

The three moving parts​

1. defer_loading: true — on a tool declaration in tools, this keeps the tool declared but withheld from the model. It is still hashed into the cache prefix (that is the whole point), but Claude never sees it as callable until you surface it.

2. tool_addition — a content block inside a role: "system" message. Surfaces a defer_loading tool from that turn onward. Also re-offers a tool that a previous tool_removal had withdrawn.

3. tool_removal — the mirror. Retracts a currently-offered tool from that turn onward. Every subsequent turn hits the cache, but the tool is no longer in Claude's choice set.

Both tool_addition and tool_removal reference a tool via a tool field. Three reference shapes are legal:

  • {"type": "tool_reference", "name": "get_forecast"} — a normal tool declared in tools.
  • {"type": "mcp_tool_reference", "server_name": "linear", "name": "create_issue"} — a single MCP connector tool.
  • {"type": "mcp_toolset_reference", "server_name": "linear"} — every tool exposed by an MCP server, in one block.

Referencing a name that is not declared in tools returns a 400.

Minimal working example​

The beta requires the mid-conversation-tool-changes-2026-07-01 header and one of Fable 5, Mythos 5, Opus 4.8, or Opus 5. Below: declare both a "read" tool and a "write" tool up front, withhold delete_file, and surface it only after the user has confirmed a destructive intent.

import anthropic

client = anthropic.Anthropic()

TOOLS = [
    {
        "name": "read_file",
        "description": "Read a file from disk.",
        "input_schema": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
        },
    },
    {
        # Declared but withheld. Hashed into the cache prefix so we can
        # surface it later without invalidating anything.
        "name": "delete_file",
        "description": "Permanently delete a file from disk.",
        "defer_loading": True,
        "input_schema": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
        },
    },
]

messages = [
    {"role": "user", "content": "Read notes.md and summarize it."},
    # ...several tool_use / tool_result turns...
    {"role": "user", "content": "OK, I confirm: delete notes.md."},
    # Surface delete_file from this point onward. The cached prefix
    # (tools + all earlier turns) still matches byte-for-byte.
    {
        "role": "system",
        "content": [
            {
                "type": "tool_addition",
                "tool": {"type": "tool_reference", "name": "delete_file"},
            }
        ],
    },
]

response = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    betas=["mid-conversation-tool-changes-2026-07-01"],
    cache_control={"type": "ephemeral"},
    tools=TOOLS,
    messages=messages,
)

The next request will:

  1. Hash tools (unchanged) → cache hit.
  2. Hash every earlier turn (unchanged) → cache hit.
  3. Only the new user turn + system-role tool_addition block are new input.

Contrast this with the "old way" — dropping delete_file into tools only at that moment. That single mutation would have invalidated the entire prefix.

Adopting it in an agentic loop​

Guided walkthrough1 of 6
  1. Include tools you plan to surface later, but set defer_loading: true on them. The point is to freeze the tool section of the cache prefix now.

Reference patterns​

Withhold destructive tools until the user confirms

system:
<tool_addition tool={type: "tool_reference", name: "delete_project"}>

Only append this after a user turn where the user explicitly confirmed destruction.
Never place before a "clarify what you want to delete?" turn.

Phase toolsets for a plan → execute → review loop

Phase 1 (plan): tools[] visible = { read_repo, search_web } — everything else defer_loading: true.
Phase 2 (execute): append system-role tool_addition for { edit_file, run_tests }.
Phase 3 (review): append system-role tool_removal for { edit_file }, tool_addition for { post_review_comment }.

The tools[] array never changes; only the offered set does. Cache is preserved across all three phases.

Retire an MCP server after a rate limit

On 429 from the Linear MCP connector, append:

system:
<tool_removal tool={type: "mcp_toolset_reference", server_name: "linear"}>

One block retires every tool that server exposed. Re-offer with a matching tool_addition once your backoff window expires.

Sandbox: give a subagent a strict subset

When you dispatch a subagent, do NOT create a new conversation with a smaller tools[]. Instead reuse the same tools[] (cache hit!) and open the subagent turn with a system-role tool_removal for every capability that subagent should not touch. The parent conversation can restore them on return with a matching tool_addition.

Placement rules (they matter — a lot)​

The role: "system" message that carries tool_addition / tool_removal blocks is a regular mid-conversation system message and inherits its placement rules:

  • Never first. A system message cannot be the first entry in messages; declare the initial toolset in the top-level system field and tools.
  • Must follow a user turn or a server-tool assistant turn. A user message carrying tool_result blocks counts — that is exactly the slot for reacting to what a tool just returned.
  • Must precede an assistant turn or be the last entry.
  • Never between a tool_use and its matching tool_result. That is a 400.

Consecutive system messages are legal and are treated as one section. You can mix tool_addition, tool_removal, and plain text blocks in the same content array.

How this interacts with prompt caching​

  • Enable caching explicitly. A cache_control field somewhere is required; automatic caching at the top level is the simplest.
  • Cache the stable prefix as usual — through the last block that does not change across requests.
  • Because the appended system message comes after the cached prefix, it does not change the prefix hash.
  • Once the system message is in the conversation, it becomes stable history and is cacheable on the next turn.
  • Every tool in tools, including defer_loading: true tools, counts toward the minimum cacheable prompt length — 512 tokens on Opus 5, 1,024 on most other models.

★ Insight ───────────────────────────────────── This design pushes agent authors toward a specific discipline: declare the ambition of the session up front, and use runtime signals to modulate access. It is closer to how OS process capabilities are modeled (capabilities you have vs. capabilities you can currently exercise) than to how classic function-calling APIs are shaped. If you architect an agent around it, "what tools does this agent have?" becomes a question with two answers — the declared universe and the offered subset — and the cache stays warm. ─────────────────────────────────────────────────

What it does not do​

  • It does not let you introduce a tool that was not in tools at all. Every tool the model may ever be offered must exist in tools from the first request. That is a feature, not a limitation — it is precisely what keeps the hash stable.
  • It does not let you change a tool's input_schema or description mid-conversation. Either of those is a mutation of tools and triggers a cache miss. If a tool's schema needs to evolve, declare two tools with different names.
  • It does not apply to Claude Sonnet 5 today. Sonnet 5 does not support mid-conversation system messages at all, so this beta cannot ride on top of it. Route Sonnet-tier turns through an outer router if you need dynamic toolsets there.

How other providers handle the same problem​

ProviderDynamic toolset without full prefix reprocess?
Anthropic Claude Opus/Fable/MythosYes, via this beta.
Anthropic Claude Sonnet 5No — resend tools (cache miss) or route through an outer supervisor.
OpenAI GPT-5/6Practically no. Changing the tools array in the Responses/Chat Completions API is a prefix change; you rely on the automatic-caching prefix match to break at the tool list. Common workaround: parent/child agents where the child has a scoped tools array.
Google Gemini 3Similar to OpenAI. The tools config is part of the request; the pragmatic pattern is per-phase Function Declaration sets, accepting the cost of re-declaring.
MCP servers in generalSome hosts (Claude Code, Cursor) implement "on-demand tool loading" inside the host, but that is transport-level: the underlying model still receives a resent tools list until this beta lands provider-side.

If you are building a cross-model harness, structure your code so the "dynamic tool" behavior is a capability you feature-detect per model rather than something you assume everywhere.

Common failure modes​

  • You forgot the beta header. The request is accepted, tool_addition / tool_removal blocks are treated as unknown content in a system message, and behavior is undefined — often the block is quietly ignored and Claude never sees the new tool.
  • You put the system message between tool_use and tool_result. 400 invalid_request_error. Move it to after the following user turn that carries the tool_result.
  • You referenced a tool not declared in tools. 400. Declare it with defer_loading: true and try again.
  • You edited the tool description "just to add clarification". Cache miss for the whole conversation. To evolve a tool mid-session, add a v2 tool under a new name and use tool_removal on v1 and tool_addition on v2.
  • You are on Sonnet 5 wondering why it does not work. It does not, on Sonnet 5. Use a different tier or an outer router.
لا توجد بطاقات بعد — أضف بعضها لبدء المذاكرة. 🃏

Check yourself

0/5
  1. Why does adding one new tool to tools[] mid-conversation invalidate every cached turn?
  2. What does defer_loading: true actually do?
  3. Where must the role: system message carrying tool_addition blocks be placed?
  4. What is the correct way to evolve a tool's input_schema mid-conversation without a full cache miss?
  5. Which Claude model does NOT support this feature today?

Sources & further reading​