Inference-Time Budget Control for LLM Search Agents
Paper: https://arxiv.org/abs/2605.05701 Authors: Zhengru Fang, Senkang Forest Hu, Zhonghao Chang, Yu Guo, Yihang Tao, Hongyao Liu, et al. (9 authors) Area: Agentic Search / Multi-hop QA
Problem and Motivation
LLM search agents increasingly rely on tools at inference time, but their trajectories are often constrained by hard limits on both tool calls and generated tokens (a *dual budget*). Under such constraints, better answers require not only stronger models, but also explicit control over which search action should receive the next budget unit and when the accumulated evidence is sufficient to commit a final answer.
Method: Two-Stage Inference-Time Budget Control
The paper studies multi-hop question answering (QA) and formulates budget control in two stages:
1. Search-time control: A controller assigns each feasible action a task-level Value-of-Information (VOI) score—an operational estimate of marginal task value per unit budget under the current search state and remaining dual budget. This score governs the choice among retrieval, decomposition, and answer commitment. 2. Answer-time control: A selective evidence-grounded finalizer compares the trajectory answer with a refined candidate and rewrites only when the residual error appears to be a low-risk answer-form error.
Results
- Evaluated on four multi-hop QA benchmarks, with three LLM backbones and four budget levels.
- The method yields positive aggregate gains over four audited baselines under the same hard dual-budget protocol.
- Ablations show that search-time budget control, especially the budget-dependent penalty, provides the main performance gain, while answer-time control helps mainly when the retrieval path is already adequate.
- Inference-time budget control should govern both how budget is spent during search and how the final answer is committed.
- The agentic paradigm turns "how many retrieval steps and which strategy" itself into a controllable, budget-aware decision process.
- For production systems, latency, cost, and answer reliability constraints make explicit budget controllers (rather than fixed trajectory limits) a practical direction.
- A Systematic Framework for Enterprise Knowledge Retrieval (arXiv 2512.05411)
- Retrieval Augmented Generation and Understanding in Vision: A Survey (arXiv 2503.18016)
- Synergizing RAG and Reasoning: A Systematic Review (arXiv 2504.15909)
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning (arXiv 2509.24193)
- Agentic Information Retrieval (arXiv 2410.09713)
Key Takeaways
Original Abstract (verbatim)
> LLM search agents increasingly rely on tools at inference time, but their trajectories are often constrained by hard limits on both tool calls and generated tokens. Under such dual budgets, better answers require not only stronger models, but also explicit control over which search action should receive the next budget unit and when the accumulated evidence is sufficient to commit a final answer. We study this problem in multi-hop question answering (QA) and formulate it as two-stage inference-time budget control. At search time, our controller assigns each feasible action a task-level Value-of-Information (VOI) score, defined as an operational estimate of marginal task value per unit budget under the current search state and remaining dual budget, and uses this score to choose among retrieval, decomposition, and answer commitment. After search, a selective evidence-grounded finalizer compares the trajectory answer with a refined candidate and rewrites only when the residual error appears to be a low-risk answer-form error. Across four multi-hop QA benchmarks, three LLM backbones, and four budget levels, the method yields positive aggregate gains over four audited baselines under the same hard dual-budget protocol. Ablations show that search-time budget control, especially budget-dependent penalty, provides the main performance gain, while answer-time control helps mainly when the retrieval path is already adequate. These results suggest that inference-time budget control for LLM search agents should govern both how budget is spent during search and how the final answer is committed.