[论文] Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality an...

论文概要

研究领域: NLP 作者: Mahdi Salmani, Adel Javanmard, Devansh Gupta, Meisam Razaviyayn 发布时间: 2026-10-22 arXiv: 2610.31667

中文摘要

测试时缩放已成为推理模型的关键推理策略——模型生成大量候选解，期望至少找到一个正确答案。但将所有候选解运行到完成的计算成本很高，因此近期研究在仅生成短前缀后就对候选池进行剪枝。现有剪枝策略大致分为两类：基于质量的方法为每个部分解打分，丢弃预测最终结果会出错的部分解；基于多样性的方法则选择遵循不同推理路径的部分解。这两种目标之间的张力本质上是探索与利用的权衡，但此前几乎没有工作直接暴露或优化这一权衡。本文提出一种通过直接优化所选解池的 pass@k 来处理该权衡的剪枝方法——它选择解池以最大化至少一个部分解在继续生成后产生正确答案的概率。由于该目标同时奖励高质量的部分解并惩罚它们之间的冗余，质量与多样性由标准本身自然地平衡，而非通过单独的临时设计选择。在八个开源推理模型上，本方法均优于现有的基于质量和基于多样性的剪枝策略。

原文摘要

Test-time scaling has emerged as a key inference strategy for reasoning models, where a model generates a large number of candidate solutions in the hope of finding at least one correct solution. Running all candidate solutions to completion can be computationally demanding, so recent work prunes the candidate pool after generating only a short prefix. Existing pruning strategies divide into quality-based and diversity-based methods. We introduce a pruning procedure that addresses this trade-off directly by optimizing pass@k over the selected solution pool. Because this objective simultaneously rewards high-quality partial solutions and penalizes redundancy among them, quality and diversity are naturally balanced by the criterion itself. On eight open-weight reasoning models, our method ou...

#论文 #arXiv #NLP #小凯