Inference-Time Budget Control for LLM Search Agents
- Paper: Inference-Time Budget Control for LLM Search Agents
- Authors: Zhengru Fang, Senkang Forest Hu, Zhonghao Chang, Yu Guo, Yihang Tao, Hongyao Liu, et al. (9 authors total)
- Date: 2026-05-07
- Link: https://arxiv.org/abs/2605.05701
- Category: Agentic Search
- Evaluated across four multi-hop QA benchmarks, three LLM backbones, and four budget levels, the method yields positive aggregate gains over four audited baselines under the same hard dual-budget protocol.
- Ablations show that search-time budget control—especially the budget-dependent penalty—provides the main performance gain.
- Answer-time control helps mainly when the retrieval path is already adequate.
- The results suggest that inference-time budget control for LLM search agents should govern both how budget is spent during search and how the final answer is committed.
Overview
LLM search agents increasingly rely on tools at inference time, but their trajectories are often constrained by hard limits on both tool calls and generated tokens. Under such dual budgets, better answers require not only stronger models, but also explicit control over which search action should receive the next budget unit and when the accumulated evidence is sufficient to commit a final answer.
The paper studies this problem in multi-hop question answering (QA) and formulates it as two-stage inference-time budget control:
1. Search time: A controller assigns each feasible action a task-level Value-of-Information (VOI) score, defined as an operational estimate of marginal task value per unit budget under the current search state and remaining dual budget. This score drives choices among retrieval, decomposition, and answer commitment. 2. Answer time: A selective, evidence-grounded finalizer compares the trajectory answer with a refined candidate and rewrites only when the residual error appears to be a low-risk answer-form error.
Key Findings
Context and Relevance
The work sits at the intersection of agentic search and large-scale search/recommendation systems, addressing how to redistribute responsibilities among retrieval, ranking, generation, and tool calling in the LLM era. Compared with classic cascaded pipelines, LLM agents introduce two new variables: the inference budget and the action space (whether to retrieve, how many times, and which tools to call). This makes budget allocation itself a first-class decision problem.
For practitioners, the paper implies that simply adding more retrieval steps is not cost-optimal under fixed budgets; explicit value estimation per action and cautious answer finalization can improve answer quality without extra budget. Open questions discussed around this line of work include evaluation reliability, latency and cost, hallucination and safety, and cross-lingual/multimodal generalization.
Citation Note
Quantitative results should be verified against the original PDF before citing: https://arxiv.org/abs/2605.05701