Summary
Autoregressive (AR) video diffusion models show strong potential for real-time video generation, and recent methods distill pretrained bidirectional video diffusion models into causal AR students via Distribution Matching Distillation (DMD). However, the reverse KL objective's mode-seeking behavior often causes the student distribution to collapse onto a few teacher modes, producing over-saturated and over-smoothed videos. This paper introduces Mask Forcing, a Dual-Noise Masking Rollout strategy that perturbs the AR student's self-rollout to mitigate mode collapse. During self-rollout in AR diffusion distillation, cleaner signals are injected into noisy rollout inputs through random masks along spatial and temporal axes. These perturbations encourage the student to explore more regions of the teacher distribution so DMD supplies learning signals beyond already-covered modes, while cleaner tokens act as denoising guidance for noisier tokens, improving intermediate predictions and reducing error accumulation. Experiments show the method improves multiple AR video diffusion distillation approaches with higher visual quality, without real video data or extra post-training stages.
Paper Overview
- Field: cs.CV
- Authors: Zhuoran Zhao, Shengju Qian, Tongtong Liang, Xianghao Kong, Songchun Zhang, Junchao Huang, Guian Fang, Xin Wang, Pan Hui, Anyi Rao
- Published: 2026-09-08
- arXiv: 2609.09123
Abstract
Autoregressive (AR) video diffusion models have shown great potential in real-time video generation. Recent methods distill pretrained bidirectional video diffusion models into causal AR students through Distribution Matching Distillation (DMD), but the generated videos often suffer from over-saturation and over-smoothing issues, resulting in limited visual quality and realism. The key contributing factor is the mode-seeking behavior of the reverse KL objective in DMD, which can cause the student distribution to collapse onto only a few modes of the teacher distribution.To address this, the authors propose Mask Forcing, a Dual-Noise Masking Rollout strategy that perturbs the AR student self-rollout to mitigate mode collapse induced by reverse-KL mode seeking.
Key Ideas
- Inject cleaner signals into noisy rollout inputs via random masks along spatial and temporal axes during the self-rollout process of AR diffusion distillation.
- These perturbations encourage the student rollouts to explore more regions of the teacher distribution, allowing DMD to provide learning signals beyond the modes already covered by the student.
- Cleaner tokens act as denoising guidance for other noisier tokens, improving intermediate rollout predictions and reducing error accumulation.
Results
Extensive experiments demonstrate that the method improves multiple AR video diffusion distillation methods with higher visual quality efficiently, without incorporating real video data or additional post-training stages.
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178634680