[论文] Learning When Diffusion Language Models Should Regenerate

论文概要

研究领域: NLP 作者: Minhae Oh, Jeonghye Kim, Jinwoo Koo, Yeongoon Kim, Jimin Seo, Minsik Seol, Jungwoo Lee 发布时间: 2026-11-12 arXiv: 2611.17320

中文摘要

重掩码（remasking）让扩散语言模型可以回溯并修改已生成的文本，但有效的纠错需要知道何时改、改什么。我们在数独任务上的诊断揭示了两大缺口：其一，隐藏状态中包含的错误信息无法被 token 置信度可靠暴露；其二，当周围上下文不变时，单独重掩码一个错误往往会原样重现。基于这些发现，我们提出 DiVR——一个验证器，预测部分生成的序列最终能否得到正确答案。DiVR 在模型自身生成轨迹的最终正确性上训练，决定何时重新生成哪个文本块、保留哪个候选，而生成器本身无需任何改动。在五个推理基准上使用 LLaDA-8B-Base，DiVR 将微调后生成器的平均准确率从 35.3% 提升至 43.4%，推理成本仅为贪心解码的 1.9-3.5 倍，优于需要 3.8-7.4 倍的搜索基线。它还能与监督微调、强化学习等后训练方法互补，并可泛化到 Dream 等其他扩散生成器。

原文摘要

Remasking allows diffusion language models to revisit generated text, but effective correction requires knowing when and what to revise. Our diagnosis on Sudoku reveals two gaps. Hidden states contain error information that token confidence does not reliably expose, and remasking an isolated error usually reproduces it when the surrounding context remains fixed. These findings motivate DiVR , a verifier that predicts whether a partially generated sequence will reach a correct answer. Trained on the final correctness of the model's own trajectories, DiVR decides when to regenerate a block and which alternative to retain, while the generator itself stays unchanged. On five reasoning benchmarks with LLaDA-8B-Base, DiVR raises the average accuracy of the fine-tuned generator from 35.3\% to 43....

#论文 #arXiv #NLP #小凯