Anthropic Tool Search Tool: Architecture, Design, and MCP Ecosystem Integration for Dynamic Tool Discovery in LLM Agents

Forum topic · ✨步子哥 · 2026-06-17

Summary

This comprehensive technical survey examines Anthropic's Tool Search Tool (TST), introduced in November 2025 as part of the advanced tool use capability set. As Model Context Protocol (MCP) ecosystems have grown to thousands of servers exposing hundreds or thousands of tools, loading all tool definitions into the context window has become prohibitively expensive and degrades tool selection accuracy. TST addresses this via dynamic tool discovery: combining deferred loading with dual-mode search (regex and BM25), the model discovers and loads only the tools needed for the current task. Published benchmarks show over 85% token reduction, roughly 95% of the context window preserved, and Opus 4 tool selection accuracy improved from 49% to 74% (Opus 4.5 from 79.5% to 88.1%). The survey compares TST against traditional full-loading, OpenAI Function Calling, programmatic tool execution via code execution (up to 98.7% token savings, complementary rather than competing), and multi-agent architectures. It details the defer_loading mechanism, prompt-caching compatibility, custom search via tool_reference blocks, and MCP integration patterns. Key limitations include incompatibility with tool use examples, search latency overhead, query-quality dependence, and platform restrictions (not on Bedrock Converse API). Future directions include native semantic search, cross-turn tool recommendation, hierarchical namespaces, and cross-provider standardization.

Anthropic Tool Search Tool: Architecture, Design, and MCP Ecosystem Integration

*Comprehensive technical survey · June 2026 — structured English summary of the original Chinese article*

Key points

The scaling problem: The MCP ecosystem has grown to thousands of servers covering productivity, development (GitHub, GitLab), communication (Slack, Discord), observability (Sentry, Grafana, Datadog), and enterprise systems (Salesforce, Jira, SAP). Loading all tool definitions into the context window incurs prohibitive token costs (e.g., a representative deployment consumed ~134,000 input tokens per request) and degrades accuracy via the "lost in the middle" / context decay effect.
The solution: The Tool Search Tool (TST), released November 2025 with the advanced-tool-use-2025-11-20 beta header, is a meta-tool inside the Claude API. Instead of preloading all tools, the model searches and discovers tools on demand via a defer_loading: true flag on tool definitions.

Performance findings

Token reduction: >85% savings; ~95% of the context window preserved. Context cost scales O(1) with tool count versus O(n) for full loading.
Accuracy gains: Opus 4 tool selection accuracy rose from 49% to 74% (+51% relative); Opus 4.5 improved from 79.5% to 88.1%. Weaker models benefit proportionally more.
Latency trade-off: full loading processed ~77,000 tokens per call vs ~8,700 with TST, plus one extra search round-trip — amortized across multi-turn sessions.

Architecture

TST is always available (never deferred); deferring *all* tools without any non-deferred tool returns HTTP 400.
Search modes: tool_search_tool_regex_20251119 (Python re.search() semantics over names, descriptions, parameters; max 200-char query) and tool_search_tool_bm25_20251119 (Okapi BM25 ranking, typically k₁=1.5, b=0.75, returning 3–5 results). Custom search implementations can return tool_reference blocks that the API expands to full definitions.
Prompt caching compatibility: the stable system-prompt prefix (TST + non-deferred tools) stays cached; discovered tool definitions are expanded into the mutable conversation body, so caching remains valid regardless of how many tools are discovered.
Discovered tools persist across the session — no re-searching is needed for already-found tools.

Comparison with alternatives

OpenAI Function Calling: also uses full loading; all definitions are sent per request. Larger windows (GPT-4.1: 1M tokens) do not solve accuracy degradation.
Programmatic tool execution (MCP code execution): model writes scripts orchestrating tools; benchmarks show 150,000 → 2,000 tokens (~98.7% savings) and 20+ calls in one script. Complementary to TST: TST solves *discovery*, code execution solves *composition*.
Multi-agent architectures: route intent to specialized agents with focused toolsets (e.g., Jenova reports 97.3% tool-use success in production); adds coordination complexity.
Traditional full loading remains appropriate for small deployments (<10 tools or <10K token definitions).

Best practices

Keep 3–5 essential tools non-deferred; write high-quality tool names/descriptions (the only indexed fields for both search backends); add a system-prompt overview of tool categories to guide query construction.
Recommended model: claude-sonnet-4-5-20250929 with max_tokens ≥ 4096. Supported on Claude API (with SSE streaming), Messages Batches API, and AWS Bedrock InvokeModel — but not Bedrock Converse API.
Zero-data-retention: search queries are not stored; usage exposes a server_tool_use.tool_search_requests counter only.

Limitations

Incompatible with tool use input_examples (which lift complex-parameter accuracy from 72% to 90%) — a real trade-off for parameter-heavy tools.
Search quality depends on query construction and description quality — a recursive problem mitigated by system-prompt guidance.
Anthropic-ecosystem only; cross-provider deployments need a unified tool-management layer.

Future directions

Native embedding-based semantic search, learned cross-turn tool recommendation/preloading, hierarchical tool namespaces, integrating examples with deferred tools, cross-provider standardization (possibly as an MCP extension), and ultimately *autonomous tool composition* — agents that discover, combine, and persist new tool capabilities as skills.

References (selected)

MCP Specification: <https://modelcontextprotocol.io/specification/>
Advanced tool use engineering post: <https://www.anthropic.com/engineering/advanced-tool-use>
Tool Search Tool docs: <https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool>
MCP code execution: <https://www.anthropic.com/engineering/code-execution-with-mcp>
Context engineering: <https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents>
Toolformer (NeurIPS 2023); Gorilla (arXiv:2305.15334); RAG (NeurIPS 2020); LLMLingua (EMNLP 2023); OpenAI Function Calling docs: <https://platform.openai.com/docs/guides/function-calling>