Pular para o conteúdo principal

Tool Search & Deferred Loading

Avançado
What you'll learn
  • Understand the one thing almost everyone gets wrong about defer_loading — it does not make your request smaller
  • Pick between the regex and BM25 search variants, and know each one's query-length ceiling
  • Read the server_tool_use / tool_search_tool_result / tool_reference block trio and know exactly what to echo back
  • Avoid the five request shapes that return a 400, including the cache_control conflict
  • Wire deferred loading into an MCP connector toolset, and replace the built-in search with your own embeddings

The two problems, and why only one of them is about money​

Loading every tool definition up front breaks in two separate ways as a catalog grows.

The first is the obvious one. Anthropic's own example: a five-server MCP setup — GitHub, Slack, Sentry, Grafana, Splunk — burns roughly 55k tokens in tool definitions before Claude does any work at all. That is a tax you pay on every turn of the conversation.

The second is the one that actually changes your architecture: tool-selection accuracy degrades once you pass 30–50 available tools. More tools does not mean more capability past that point — it means a model choosing from a crowded menu. Deferred loading fixes the cost problem and the accuracy problem with the same move, because the model only ever sees the 3–5 tools relevant to the current request. Anthropic reports the context reduction as over 85 percent on that multiserver setup.

If you want the economics and the published accuracy deltas, AILmanac covers them in MCP token cost. This page is about the wiring.

The mental model people get wrong​

Here is the misconception worth correcting before you write any code:

defer_loading does not reduce what you send. You still put every tool's full definition — names, descriptions, complete input_schema — in the tools array on every single request, deferred ones included.

Your HTTP request body does not shrink. Your upload bandwidth does not change. The API needs those definitions server-side so it can run the search and expand the references it finds.

What changes is what enters the context window, and therefore what you are billed for. Deferred definitions are excluded from the system-prompt prefix. When Claude discovers one, the API appends a tool_reference block inline in the conversation and expands it into the full definition before the model sees it.

That inline-append detail is the second non-obvious thing: because the expansion happens in the conversation and not in the prefix, the cached prefix is never invalidated. Deferred loading and prompt caching compose. So does strict tool use — the grammar that constrains tool-call output is built from the full toolset, so nothing gets recompiled when a tool arrives mid-conversation.

Turning it on​

Guided walkthrough1 of 5
  1. Pick one variant: tool_search_tool_regex_20251119 or tool_search_tool_bm25_20251119. It is a server tool — it runs on Anthropic's infrastructure, and it needs no beta header.

Minimal working request — one search tool, two deferred tools

curl https://api.anthropic.com/v1/messages \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
  "model": "claude-opus-5-5",
  "max_tokens": 2048,
  "messages": [{"role": "user", "content": "What is the weather in San Francisco?"}],
  "tools": [
    {
      "type": "tool_search_tool_regex_20251119",
      "name": "tool_search_tool_regex"
    },
    {
      "name": "get_weather",
      "description": "Get the weather at a specific location",
      "input_schema": {
        "type": "object",
        "properties": {
          "location": {"type": "string"},
          "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
        },
        "required": ["location"]
      },
      "defer_loading": true
    },
    {
      "name": "search_files",
      "description": "Search through files in the workspace",
      "input_schema": {
        "type": "object",
        "properties": {
          "query": {"type": "string"},
          "file_types": {"type": "array", "items": {"type": "string"}}
        },
        "required": ["query"]
      },
      "defer_loading": true
    }
  ]
}'

Note what the search tool costs you in the prefix: nothing but its own entry. The two deferred definitions are in the body but not in context.

Regex or BM25​

Both variants search the same four fields — tool names, descriptions, argument names, and argument descriptions — and both are available on every model that supports the feature. They differ only in what Claude writes into the query.

tool_search_tool_regex_20251119tool_search_tool_bm25_20251119
Claude writesPython re.search() patternsnatural-language queries
Input fieldpatternquery
Max length200 characters500 characters
Matchingcase-insensitiveBM25 ranking
Good whenyour names are namespaced (github_, slack_)your descriptions read like prose

The regex variant is the one people misconfigure, because the name suggests a keyword search and it is not. Claude writes real patterns: "weather", "get_.*_data", "database.*query|query.*database". If the model is not finding a tool you expect, test the pattern yourself before blaming the model:

Reproduce a failed regex match locally

import re
# Claude writes broad patterns, e.g. ".*weather.*" — not exact names.
# Search the same four fields the API searches.
haystacks = ["get_weather", "Get the weather at a specific location", "location", "unit"]
print(any(re.search(r"weather", h, re.IGNORECASE) for h in haystacks))

The practical fix for a missed match is almost never a cleverer pattern — it is keywords in the description. Write the words your users would use. Namespace your tool names by service so one pattern sweeps a whole group.

Reading the response​

A successful search produces three new block types in the assistant turn, in this order:

{
  "role": "assistant",
  "content": [
    { "type": "text", "text": "I'll search for tools to help with the weather information." },
    {
      "type": "server_tool_use",
      "id": "srvtoolu_01ABC123",
      "name": "tool_search_tool_regex",
      "input": { "pattern": "weather", "limit": 10 }
    },
    {
      "type": "tool_search_tool_result",
      "tool_use_id": "srvtoolu_01ABC123",
      "content": {
        "type": "tool_search_tool_search_result",
        "tool_references": [{ "type": "tool_reference", "tool_name": "get_weather" }]
      }
    },
    {
      "type": "tool_use",
      "id": "toolu_01XYZ789",
      "name": "get_weather",
      "input": { "location": "San Francisco", "unit": "fahrenheit" }
    }
  ],
  "stop_reason": "tool_use"
}

Three rules for handling it:

  1. Never return a tool_result for the srvtoolu_... ID. The search already ran on Anthropic's servers. Reply to it and the API rejects the request. Your tool_result goes to the toolu_... ID only — ordinary tool use from there.
  2. Never expand tool_reference blocks yourself. The API does it, as long as the matching definition is present in tools.
  3. Keep tool_search_tool_result in the history as is. It is how later turns know which tools are already discovered.

A search that matches nothing is not an error — you get a tool_search_tool_search_result with an empty tool_references array, and Claude decides what to do next.

One quiet detail on limit: it defaults to 5 results, but Claude can set anything from 1 to 10,000 in its own search input. You do not control it directly. If you need a hard ceiling on how many definitions can flood in, that is an argument for the custom implementation below.

The five 400s​

Four of these are request-shape errors that happen before any inference, and one is the conflict nobody expects.

Guided walkthrough1 of 5
  1. "At least one tool must have defer_loading=false. All tools cannot be deferred." The usual cause is a loop that stamps defer_loading: true over the whole array, search tool included. Never defer the search tool.

Search failures during execution are different: they come back with HTTP 200 and the error inside a tool_search_tool_result_error object.

error_codeWhat happened
invalid_tool_inputMalformed regex, or a pattern over the 200-character limit
unavailableThe search could not run — timeout or service unavailable
too_many_requestsRate limit on tool search operations specifically
execution_time_exceededThe search blew its execution-time limit

Treat these as recoverable: the conversation is intact, and the honest move is usually to let Claude retry with a different pattern rather than failing the whole turn. See Errors & rate limits for the surrounding retry policy.

Prompt caching​

The headline is good: deferred definitions live outside the cached prefix, so adding and discovering tools mid-conversation does not invalidate it. See Prompt caching for the prefix rules, and Cache diagnostics if you need to prove a hit.

The trap is the one listed above — defer_loading: true and cache_control are mutually exclusive on a single tool. If your breakpoint currently sits on the last tool in a long array and you then defer that array, you have quietly moved from "cached" to "400". Anchor the breakpoint on a tool you were always going to keep loaded.

MCP connector: set it on the toolset, not the tool​

With tools arriving from remote servers through the MCP connector, you do not touch individual definitions — you do not own them. Configure it on the mcp_toolset entry: once for the whole server via default_config, or per tool in configs.

Defer a whole MCP server except the one tool you always need

{
"type": "mcp_toolset",
"mcp_server_name": "observability",
"default_config": { "defer_loading": true },
"configs": {
  "search_events": { "defer_loading": false }
}
}

This is the shape that pays off fastest in practice: aggregating several servers is exactly the 200-plus-tool situation the feature exists for, and you almost always have one or two entry-point tools worth keeping hot.

Replacing the search with your own​

Both built-in variants are lexical. If your tools are better found by meaning than by wording, implement the search yourself: define an ordinary custom tool, and return tool_reference blocks in a normal tool_result.

A client-side search result the API will expand

{
"type": "tool_result",
"tool_use_id": "toolu_your_tool_id",
"content": [
  { "type": "tool_reference", "tool_name": "discovered_tool_name" }
]
}

Every name you reference still needs its full definition in the top-level tools array, normally with defer_loading: true. The expansion path is identical — you have only swapped out the retrieval. Note the format difference: tool_search_tool_result is the server-side shape, internal to the built-in variants. Your custom tool returns the standard tool_result shape shown above. Anthropic publishes a worked embeddings recipe.

This is also your escape hatch for the limit problem: your own search decides how many references come back.

When to skip it​

Deferred loading buys you nothing — and costs you a search round-trip — when the premise does not hold. Reach for it when any of these are true:

  • 10 or more tools available
  • tool definitions above roughly 10k tokens
  • selection accuracy visibly dropping as the catalog grows
  • multiple MCP servers aggregated (200-plus tools)
  • a tool library that keeps growing

Stay with plain tool calling when you have fewer than 10 tools, or every tool is used on every request, or the whole catalog is under about 100 tokens. And note the billing shape: tool search is not metered as a separate server tool. There is no tool-search field in usage.server_tool_use. The only tokens it costs are the definitions it pulls into context, priced like any other tool definition.

The hard ceilings, for capacity planning: 10,000 deferred tools per request, 200-character regex patterns, 500-character BM25 queries.

Platform caveats​

Two deployment footnotes that are easy to lose a day to. On Amazon Bedrock, server-side tool search works only through the InvokeModel API — not the Converse API. On Claude Platform on AWS it behaves exactly like the Claude API, because that path uses the Messages API directly and has no InvokeModel/Converse split. Claude Opus 4.1 and earlier do not support the feature at all.

Tool search vocabulary
Pressione Enter ou Espaço para virar o cartão. Use as setas esquerda e direita para navegar entre os cartões.Termo exibido.
1 / 8

Check yourself

0/6
  1. You set defer_loading: true on 400 of your 405 tools. What happens to the size of the JSON you POST?
  2. A loop stamps defer_loading: true across your whole tools array. What does the API return?
  3. Your response contains a server_tool_use block with id srvtoolu_01ABC and a tool_use block with id toolu_01XYZ. What do you send back?
  4. You already had a cache breakpoint on the last tool in your array. You now defer that array. What breaks?
  5. Claude keeps failing to find your Grafana tools with the regex variant. Best first move?
  6. How is tool search billed?

Sources & further reading​

Next​

  • Programmatic Tool Calling — the other half of advanced tool use: search finds the right tool, programmatic calling executes it without dragging results through context.
  • MCP token cost — the economics and the published accuracy deltas behind this feature.
  • Memory & context editing — what to do about the results filling your context, once the definitions are handled.