Summary
RAVEN (Real-time Autoregressive Video Extrapolation Network) is a new framework for real-time streaming video generation with causal autoregressive video diffusion models. These models generate future video segments by extrapolating from previously generated content, and can be distilled from high-fidelity bidirectional teacher models into competitive few-step generators. However, a persistent gap between the historical distributions seen during training and those produced at inference limits long-horizon generation quality. RAVEN addresses this with a test-time-aware training framework that repackages each self-bootstrapped rollout into an interleaved sequence of clean historical endpoints and noisy denoising states, aligning training attention with inference-time extrapolation and letting downstream segment losses supervise the historical representations that future predictions depend on. The authors further propose CM-GRPO (Consistency Model Group Relative Policy Optimization), which reformulates consistency sampling steps as conditional Gaussian transitions and applies online reinforcement learning directly to this kernel, avoiding the Euler-Maruyama auxiliary processes used in prior flow-model RL formulations. Experiments show RAVEN surpasses recent causal video distillation baselines on quality, semantic, and dynamics evaluations, with CM-GRPO providing additional gains when combined with RAVEN. Paper: arXiv 2605.15190, by Yanzuo Lu, Ronglai Zuo, and Jiankang Deng.
Paper Overview
Field: Computer Vision (CV)
Authors: Yanzuo Lu, Ronglai Zuo, Jiankang Deng
Published: 2026-05-14
arXiv: 2605.15190
Abstract
Causal autoregressive video diffusion models enable real-time streaming generation by extrapolating future segments from previously generated content. Distilling such generators from high-fidelity bidirectional teacher models yields competitive few-step models, but a persistent gap between the historical distributions encountered during training and those produced at inference limits long-horizon generation quality.
We present the Real-time Autoregressive Video Extrapolation Network (RAVEN), a test-time-aware training framework that repackages each self-bootstrapped rollout into an interleaved sequence of clean historical endpoints and noisy denoising states. This formulation aligns training attention with inference-time extrapolation and allows downstream segment losses to supervise the historical representations on which future predictions depend.
We further propose Consistency Model Group Relative Policy Optimization (CM-GRPO), which reformulates consistency sampling steps as conditional Gaussian transitions and applies online reinforcement learning (RL) directly to this kernel, avoiding the Euler-Maruyama auxiliary processes adopted in prior flow-model RL formulations.
Experiments show that RAVEN surpasses recent causal video distillation baselines in quality, semantic, and dynamics evaluations, and that CM-GRPO provides further gains when combined with RAVEN.
---
*Automatically collected on 2026-05-17*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177620161