Paper Overview
Field: NLP Authors: Minhae Oh, Jeonghye Kim, Jinwoo Koo, Yeongoon Kim, Jimin Seo, Minsik Seol, Jungwoo Lee Published: 2026-11-12 arXiv: 2611.17320
Abstract (translated)
Remasking allows diffusion language models to revisit generated text, but effective correction requires knowing when and what to revise. The authors' diagnosis on Sudoku reveals two gaps: hidden states contain error information that token confidence does not reliably expose, and remasking an isolated error usually reproduces it when the surrounding context remains fixed. These findings motivate DiVR, a verifier that predicts whether a partially generated sequence will reach a correct answer. Trained on the final correctness of the model's own trajectories, DiVR decides when to regenerate a block and which alternative to retain, while the generator itself stays unchanged.
Key Results
- On five reasoning benchmarks with LLaDA-8B-Base, DiVR raises the average accuracy of the fine-tuned generator from 35.3% to 43.4%.
- Inference cost is only 1.9-3.5x greedy decoding, compared with 3.8-7.4x for search baselines.
- DiVR is complementary to post-training methods such as supervised fine-tuning and reinforcement learning.
- The approach generalizes to other diffusion generators, including Dream.
Discussion
The core insight is that a verifier trained on trajectory outcomes can guide remasking decisions better than token-level confidence, since confidence scores fail to reflect latent errors. By regenerating whole blocks rather than isolated tokens, DiVR avoids the failure mode where a corrected token simply reverts under unchanged context.
--- *Auto-collected on 2026-11-14.*