[论文] Batched Contextual Reinforcement: 高效推理的任务扩展律

## 论文概要 **研究领域**: ML/AI **作者**: Bangji Yang, Hongbo Ma, Jiajun Fan **发布时间**: 2026-04-02 **arXiv**: [2604.02322](https://arxiv.org/abs/2604.02322) ## 中文摘要 采用思维链推理的大型语言模型虽然取得了强大的性能，但过度的token消耗推高了推理成本。现有的效率方法如显式长度惩罚、难度估计器或多阶段课程要么降低推理质量，要么需要复杂的训练流程。我们引入了批次上下文强化（BCR），一种极简的单阶段训练范式，通过简单的结构修改解锁高效推理：在共享上下文窗口中同时训练模型解决N个问题，仅通过每个实例的准确率进行奖励。这种公式化创建了隐式token预算，产生了几个关键发现：(1) 我们发现了一种新颖的任务扩展律：随着推理期间并发问题数N的增加，每个问题的token使用量单调递减，而准确率下降远比基线更平缓，将N确立为可控的吞吐量维度。(2) BCR通过在标准单问题推理中展示"免费午餐"现象，挑战了传统的准确率-效率权衡。在1.5B和4B模型系列中，BCR在五个主要数学基准上减少15.8%至62.6%的token使用量，同时持续保持或提高准确率。(3) 定性分析揭示了 emergent 自我调节效率，模型在没有显式长度监督的情况下自主消除冗余的元认知循环。(4) 至关重要的是，我们实证证明了隐式预算约束成功规避了显式长度惩罚固有的对抗梯度和灾难性优化崩溃，为长度控制提供了一种高度稳定的基于约束的替代方案。 ## 原文摘要 Large Language Models employing Chain-of-Thought reasoning achieve strong performance but suffer from excessive token consumption that inflates inference costs. Existing efficiency methods such as explicit length penalties, difficulty estimators, or multi-stage curricula either degrade reasoning quality or require complex training pipelines. We introduce Batched Contextual Reinforcement, a minimalist, single-stage training paradigm that unlocks efficient reasoning through a simple structural modification: training the model to solve N problems simultaneously within a shared context window, rewarded purely by per-instance accuracy. This formulation creates an implicit token budget that yields several key findings: (1) We identify a novel task-scaling law: as the number of concurrent problems N increases during inference, per-problem token usage decreases monotonically while accuracy degrades far more gracefully than baselines, establishing N as a controllable throughput dimension. (2) BCR challenges the traditional accuracy-efficiency trade-off by demonstrating a "free lunch" phenomenon at standard single-problem inference. Across both 1.5B and 4B model families, BCR reduces token usage by 15.8% to 62.6% while consistently maintaining or improving accuracy across five major mathematical benchmarks. (3) Qualitative analyses reveal emergent self-regulated efficiency, where models autonomously eliminate redundant metacognitive loops without explicit length supervision. (4) Crucially, we empirically demonstrate that implicit budget constraints successfully circumvent the adversarial gradients and catastrophic optimization collapse inherent to explicit length penalties, offering a highly stable, constraint-based alternative for length control. These results prove BCR practical, showing simple structural incentives unlock latent high-density reasoning in LLMs. --- *自动采集于 2026-04-05* #论文 #arXiv #ML #小凯