[论文] Acceptance Is Not Speed: Length Inflation in Training-Free Lossy Specu...

论文概要

研究领域: NLP 作者: Chiatzen Wang, Bill Xu, Haochen Gu, Hongyang Zhang 发布时间: 2026-10-16 arXiv: 2610.23818

中文摘要

投机解码通常以每轮验证接受的草稿 token 数来刻画加速比。无损验证保持目标模型的输出分布和期望生成长度不变，因此该指标是有意义的效率代理。松弛（有损）验证接受无损验证会拒绝的 token，可能改变该分布。我们发现了一种长度膨胀现象：松弛验证使生成结果变长，主要表现为推理痕迹的延长，这在很大程度上抵消了每轮接受 token 数增加带来的表面收益。我们在 12 组目标-草稿模型配对上评估了现有的免训练松弛验证规则，与相同提示下的无损验证进行对比，测量生成长度、验证轮数、实际耗时和任务准确率。在各规则最宽松的评估设定下，松弛验证平均每轮多接受 1.34 倍的 token，但生成结果长 1.54 倍。对于接受长度更高的规则，仅在松弛验证下被接受的 token 往往具有较低的目标模型概率，促成了长度膨胀。对于减少验证轮数的规则，准确率通常大幅下降。进一步分析将部分失败追溯到持续的推理耗尽了交付最终答案所需的预算。总体而言，仅报告每轮接受 token 数在松弛验证下不可靠——还必须报告生成长度和验证轮数才能评估实际效率。

原文摘要

Speculative decoding typically characterizes speedup by draft tokens accepted per verification round. Lossless verification preserves the target model's output distribution and thus expected completion length, making this metric a meaningful efficiency proxy. Relaxed (lossy) verification accepts tokens that lossless verification would reject, potentially altering that distribution. We identify a length inflation phenomenon: relaxed verification lengthens completions, primarily by extending reasoning traces, largely offsetting apparent gains in accepted tokens per round. We evaluate existing training-free relaxed verification rules on twelve target-drafter pairs against lossless verification on identical prompts, measuring completion length, verifier rounds, wall-clock time, and task accura...

#论文 #arXiv #NLP #小凯