Paper Overview
Research Area: Computer Vision (CV) Authors: Yanke Song, Shuo Chen, Chris Rockwell, Guande He, Nan Liu, Desai Xie, Tianpei Gu, Yong-Hyun Park, Aryan V S, Dennis Yuren Menn, Ming-Hsuan Yang, Xun Huang Published: 2026-11-12 arXiv: 2611.17336
Abstract
Frontier video generation models work in a VAE latent space that compresses video in both space and time. This compression made training tractable, but now limits high-resolution, controllable, and interactive generation — doubling resolution quadruples the token count, conditions apply to groups of frames rather than individual ones, and each response must wait for the rest of its multi-frame block. We present PixVid, a simple yet effective method that lifts a pretrained latent video model to directly generate every frame in pixel space. PixVid keeps the diffusion transformer and replaces the VAE decoder with a convolutional head that renders its outputs into pixels. We find that predicting clean video rather than velocity, together with a larger noise shift, is key to quality. Fine-tuned from Wan2.1-14B with only about 1.3 million training clips, PixVid reaches parity with the latent baseline. Working frame-by-frame in pixel space, it generates at 2x higher resolution with the same per-frame token count, preserves alignment for depth-conditioned video editing under fast motion, and responds instantly to each action on the next frame.
Key Results
- Direct pixel-space, per-frame generation from a pretrained latent video model
- Convolutional head replaces the VAE decoder; diffusion transformer retained
- Clean-video prediction + larger noise shift are critical to quality
- ~1.3M training clips fine-tuned from Wan2.1-14B
- 2x higher resolution at identical per-frame token count; per-frame conditioning and next-frame interactivity
--- *Auto-collected on 2026-11-14*