[论文] PORA: Learning When More Reasoning Pays Off for LLM Agents

论文概要

研究领域: ML
作者: Bo Yan, Qi Zhang, Zhen Tan, Tianlong Chen, Song Wang
发布时间: 2026-10-22
arXiv: 2610.27119

中文摘要

LLM智能体在多步执行任务并根据反馈修正动作。由于总推理预算通常有限且必须支持后续步骤，智能体需要在每个动作前选择短推理还是长推理。更长的推理可能通过预防后续错误来改善最终任务结果，但它也会占用未来决策可能需要的预算容量。这带来了两个挑战：估计长推理的下游价值，以及决定该价值是否值得使用剩余预算。我们提出PORA（配对结果引导的推理分配），通过比较同一交互状态下的短推理和长推理来学习这一决策。训练过程中，PORA从每个选定状态完成任务两次——一次使用短推理额度，一次使用长推理额度——比较它们的最终质量和总处理量，以估计何时更长的推理更有价值。推理过程中，控制器将此估计与对剩余预算压力的学习价格进行权衡，选择短推理或长推理，而无需同时运行两种替代方案。我们在应用、记忆、代码、零售和数学任务上评估PORA，在所有设置下均取得强劲结果。在BFCL记忆任务上，它超过最强现有方法17.4-32.9个百分点。在同一智能体内，它在代码和数学任务上匹配或超过始终使用长推理的效果，同时分别减少50.6%的智能体token和31.2%的模型请求。

原文摘要

LLM agents perform tasks over multiple steps and revise their actions based on feedback. As the total reasoning budget is often limited and must support future steps, the agent must choose between short and long reasoning before each action. Longer reasoning may improve the final task outcome by preventing later errors, but it also uses budget capacity that may be needed for future decisions. This creates two challenges: estimating the downstream value of long reasoning and deciding when that value is worth using the remaining budget. We propose Paired Outcome-guided Reasoning Allocation (PORA), which learns this decision by comparing short and long reasoning from the same interaction state. During training, \method{} completes the task twice from each selected state, once with a short rea...

自动采集于 2026-10-22

#论文 #arXiv #ML #小凯