Key points
Paper: RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO
- Authors: Yanzuo Lu, Ronglai Zuo, Jiankang Deng
- arXiv: https://arxiv.org/abs/2605.15190
- Field: computer vision / video generation
- In real-time autoregressive video generation, the model must generate frame chunks on the fly, conditioning each new chunk on its own previously generated output.
- Existing methods either train on ground-truth history (which differs from the model's own rollout distribution at inference) or condition on rollout history without end-to-end supervision — the "history supervision gap."
- During autoregressive extrapolation, small deviations compound like repeated photocopying, degrading quality over long generations.
- Naively making training autoregressive would require backpropagating through an entire autoregressive sampling trajectory (dozens of chunks × many denoising steps) — computationally infeasible.
- RAVEN's insight: during a fake-score training step, the model already performs a full self-rollout. Rather than re-simulating inference, RAVEN repackages these existing states into a single long sequence that interleaves clean generated chunks (used as attention history context) with noisy denoising states (the supervised targets).
- This allows one forward pass over multiple chunks, and gradients from later chunks flow back to shape the cached history representations that future predictions depend on — end-to-end supervision without recursive backpropagation through hundreds of steps.
- Consistency models learn a near one-step mapping from noise to clean output, enabling real-time sampling, but their deterministic sampling seems incompatible with exploration-based RL like GRPO (Group Relative Policy Optimization, from DeepSeek-R1), which compares a group of sampled answers without a separate reward model.
- Prior work (Flow-GRPO) converts the deterministic ODE into an SDE via Euler-Maruyama, but this optimizes stochastic transitions that differ from deterministic inference — another train-test discrepancy.
- CM-GRPO reformulates each consistency-model sampling step as a conditional Gaussian transition:
z_next = α · x̂ + σ · ε, where x̂ is the predicted clean endpoint. This kernel is already used at inference, so training and inference stay aligned — especially important in autoregressive video generation, where each chunk alters the history for all subsequent predictions. - RAVEN outperforms baselines such as CausVid and Causal Forcing on all dimensions.
- Ablations vs. Euler-Maruyama auxiliary processes: best EM configuration reaches only 85.27, confirming that applying RL directly on the consistency model's own sampling kernel yields measurable gains from train-test consistency.
- Reward ablations: dynamic degree reward is the main driver of motion supervision; over-weighting aesthetics and imaging quality improves visuals but suppresses motion — a persistent tension between "looking good" and "actually moving."
- The train/inference mismatch mirrors "exposure bias" in language models, but is harder in video diffusion due to multi-step denoising.
- RAVEN aligns training-time history with inference-time history at no extra rollout cost, and CM-GRPO aligns RL optimization with the true sampling kernel — together advancing quality for streaming, real-time video generation (VR worlds, live avatars, generative game engines).
The problem: the history supervision gap
RAVEN's solution: repackage the self-rollout
CM-GRPO: RL on consistency models without extra stochasticity
Experimental results (Wan2.1-T2V-1.3B, VBench)
| Method | Total | Quality | Semantic | Dynamic Degree | |---|---|---|---|---| | RAVEN | 85.15 | 86.18 | 81.04 | 2.951 | | RAVEN + CM-GRPO | 85.46 | 86.54 | 81.17 | 2.962 |