Paper Overview
- Field: Machine Learning
- Authors: Yize Cheng, Chenrui Fan, Mahdi JafariRaviz
- Published: 2026-05-17
- arXiv: 2505.12354
- Tool use is decomposed into two stages:
- An internal cognition stage — whether the model believes a tool is needed.
- An execution stage — whether the model actually makes the tool-call action.
- Probing LLM hidden states shows both signals are typically linearly decodable, but their probing directions become nearly orthogonal in the deeper, last-token regions that drive next-token actions.
- Tracing sample trajectories through the two-stage process shows most mismatches concentrate in the cognition-to-action transition, not in cognition itself.
Summary
Large language models (LLMs) increasingly act as autonomous agents that must decide when to answer directly vs. when to invoke external tools. Prior work studying adaptive tool use has largely treated tool necessity as a model-agnostic property, annotated by human or LLM judge, and mostly covers cases where the answer is obvious (e.g., fetching the weather vs. paraphrasing text). However, tool necessity in the wild is more nuanced due to the divergence of capability boundaries across models: a problem solvable by a strong model on its own may still require tools for a weaker one.
The authors introduce a model-adaptive definition of tool necessity, grounded in each model's empirical performance. Under this definition, they compare necessity against observed tool-call behavior across four models on arithmetic and factual QA datasets, finding significant mismatches of 26.5–54.0% and 30.8–41.8% respectively.
Key Findings
Conclusion
These results reveal a knowing-doing gap in LLM tool use: improving tool-use reliability requires not only better recognition of when tools are needed, but also better translation of that recognition into action.
---
*Auto-collected on 2026-05-18*