Dynamic Tool Discovery for LLMs: Anthropic's Tool Search Tool Architecture, Design, and MCP Ecosystem Integration

Forum topic · ✨步子哥 · 2026-06-17

Summary

This comprehensive technical survey examines Anthropic's Tool Search Tool (TST), introduced in November 2025 as part of the advanced tool use capability set for the Claude API. As LLM agents connect to hundreds or thousands of tools across Model Context Protocol (MCP) servers, loading all tool definitions into the context window incurs prohibitive token costs and degrades tool-selection accuracy. TST addresses this scaling crisis through lazy loading (`defer_loading` flag) combined with dual-mode search (regex-based and BM25-based), enabling models to discover and load only task-relevant tools on demand. Published benchmarks show TST reduces token consumption by over 85%, preserves roughly 95% of the context window, and improves Opus 4's tool-selection accuracy from 49% to 74% (a 51% relative gain). The paper compares TST against OpenAI Function Calling, traditional eager loading, programmatic tool calling via code execution (up to 98.7% token savings), and multi-agent architectures, positioning TST as complementary to code execution and agent routing. Key architectural features include prompt-caching compatibility, tool persistence across conversation turns, custom search extensions via `tool_reference` blocks, and streaming/batch API support. Limitations include incompatibility with tool use examples, search-latency overhead, query-quality dependence, and platform restriction to the Anthropic ecosystem. Future directions cover semantic search, hierarchical tool namespaces, cross-turn recommendations, and cross-provider standardization.

Dynamic Tool Discovery: Anthropic's Tool Search Tool — Architecture, Design, and Ecosystem Integration

*Comprehensive technical survey · June 2026*

Key points

The scaling crisis: MCP's rapid adoption (thousands of servers) created a paradox — agents become more capable with more tools, yet harder to use effectively. Tool definitions scale linearly in token cost (a representative deployment can consume ~134,000 input tokens per request), and accuracy degrades via context rot and the "lost in the middle" effect.
TST's approach: A meta-tool inside the Claude Messages API. Tools marked defer_loading: true are excluded from the initial system prompt and stored in a deferred registry; the model searches on demand via regex (tool_search_tool_regex_20251119) or BM25 (tool_search_tool_bm25_20251119), and returned tool_reference blocks are expanded into full definitions in the conversation body.
Measured results: >85% token reduction, ~95% context window preserved, and tool-selection accuracy improved from 49% → 74% on Opus 4 (a 25-point gain); even Opus 4.5 (79.5% baseline) gains 8.6 points. Context cost is effectively O(1) vs. O(n) for eager loading.
Architecture properties: TST is always available (never deferred); a hard constraint requires at least one non-deferred tool (all-deferred returns HTTP 400). Because deferred definitions never appear in the stable prefix, prompt caching remains fully valid. Discovered tools persist across turns via automatic expansion throughout conversation history.

Comparison of tool-management approaches

| Approach | Mechanism | Trade-offs | |---|---|---| | Traditional eager loading | All definitions preloaded | Simple, but token-prohibitive at scale; still fine for <10 tools / <10K tokens | | OpenAI Function Calling | Eager loading of all tools each request; larger context windows (up to 1M tokens) | Does not solve fundamental scaling/accuracy limits; separates tools from tool_choice | | Programmatic tool calling (code execution) | Tools exposed as code APIs; intermediate results stay in the execution environment | Up to 98.7% token savings (150K → 2K) and fewer inference round-trips, but requires a secure execution environment; complementary to TST (composition vs. discovery) | | Multi-agent architectures | Specialized sub-agents with focused tool sets (e.g., Jenova reports 97.3% tool-use success) | Adds routing/coordination complexity and cost; orthogonal to TST |

Search mechanics

Regex search: Python re.search() semantics against tool names, descriptions, parameter names, and descriptions; queries capped at 200 characters; deterministic and precise.
BM25 search: Okapi BM25 (typical k₁=1.5, b=0.75) with length normalization and IDF weighting; accepts natural-language queries (e.g., "send a message to a channel" matches post_message, send_notification); returns top 3–5 results.
Custom search: Developers can implement embedding-based or domain-specific search returning tool_reference blocks; every referenced tool must exist with defer_loading: true.
Quality bottleneck: Only names and descriptions are indexed, so documentation quality is a hard ceiling on search quality.

Integration and best practices

MCP integration: Entire servers can be deferred, with a few high-frequency tools selectively non-deferred, enabling progressive disclosure mirroring human information-seeking.
Skills: Reusable instruction/script folders can encapsulate tool orchestration patterns, reducing repeated discovery (e.g., a "deploy to production" skill pre-packaging GitHub/Slack/Jira workflows).
Implementation: Activated via the beta header advanced-tool-use-2025-11-20; recommended model claude-sonnet-4-5-20250929, min_tokens ≥ 4096 (max_tokens recommended at least 4096). Full SSE streaming support with server_tool_use event types; supported in the Messages Batches API; supported on Bedrock via InvokeModel (not Converse API). Zero-data-retention customers' search queries are not stored.
Best practices: Keep 3–5 essential tools non-deferred; write high-quality names/descriptions; add a system-prompt overview of tool categories (e.g., "tools for Slack (messaging, channels), GitHub (repos, issues, PRs), Jira (tickets)... — use the Tool Search Tool when you need a specific tool from these categories"); adopt via phased migration.

Limitations

Incompatible with tool use examples (input_examples): deferred tools' examples are not expanded; since examples raise complex-parameter accuracy from 72% → 90%, critical complex tools may be better kept non-deferred.
Latency overhead: extra round-trip for discovery, partly offset by smaller contexts and amortized over multi-turn sessions.
Query-quality dependence: a recursive problem — the model needs ecosystem understanding to write good queries; system-prompt category hints are the main mitigation.
Platform lock-in: Anthropic-ecosystem only; cross-provider deployments need a unified tool-management layer.

Future directions

1. Native embedding-based semantic search (matching "upload" with "create file", multilingual queries). 2. Cross-turn/cross-session tool recommendation and proactive preloading (with privacy considerations). 3. Hierarchical tool namespaces with category filters for 10,000-tool scale. 4. Deferred expansion of tool use examples. 5. Cross-provider standardization, possibly as an MCP extension. 6. Autonomous tool composition: agents that discover, combine, and save new tool combinations as skills — from "tool-using" to "tool-creating" agents.

Related work

TST builds on tool-augmented LLMs (Toolformer, Gorilla), shares foundations with retrieval-augmented generation (tool definitions as retrieved "documents" expanded into callable functions), complements context-window optimization (LLMLingua-style compression risks losing parameter details, whereas TST preserves full fidelity of loaded tools), and differs from agent frameworks (LangChain, AutoGPT) by embedding discovery inside the LLM inference loop.

Conclusion

TST marks an architectural shift from "load everything" to "discover on demand": with >85% token reduction and a 51% relative accuracy gain for Opus 4, it delivers dynamic tool discovery with minimal operational overhead and full backward compatibility, while its remaining limitations (examples incompatibility, query-quality dependence) define active areas for improvement.

References

1. Anthropic, "Model Context Protocol Specification," 2024. <https://modelcontextprotocol.io/specification/> 2. A. Jones and C. Kelly, "MCP Code Execution: Building More Efficient Agents," Anthropic Engineering Blog, Nov 2025. <https://www.anthropic.com/engineering/code-execution-with-mcp> 3. Anthropic, "Introducing Advanced Tool Use on the Claude Developer Platform," Nov 2025. <https://www.anthropic.com/engineering/advanced-tool-use> 4. Anthropic, "Effective Context Engineering for AI Agents," 2025. <https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents> 5. Anthropic, "Tool Search Tool — Claude Platform Docs." <https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool> 6. T. Schick et al., "Toolformer: Language Models Can Teach Themselves to Use Tools," NeurIPS, 2023. 7. S. G. Patil et al., "Gorilla: Connecting LLMs with Massive APIs," arXiv:2305.15334, 2023. 8. P. Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," NeurIPS, 2020. 9. H. Jiang et al., "LLMLingua: Compressing Prompts for Accelerated Inference," EMNLP, 2023. 10. LangChain. <https://github.com/langchain-ai/langchain> 11. Anthropic, "Deep Dive into Claude Code," arXiv:2604.14228, 2026. 12. Jenova, "How Jenova Solved the AI Tool Scalability Problem," MintMCP Blog, Nov 2025. <https://www.mintmcp.com/blog/how_jenova_solved_the_ai_tool_scalability_problem> 13. X. Liu et al., "MemTool: Short-Term Memory Management for Dynamic Tool Calls," 2025. 14. OpenAI, "Function Calling — OpenAI API Docs." <https://platform.openai.com/docs/guides/function-calling> 15. "Tool Use vs. Function Calling: A Deep Comparison of LLM Tool-Calling Architectures," CSDN, April 2026.