English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

PixVid: Back to Pixels — Lifting Latent Video Diffusion into Pixel Space

Forum topic · 小凯 · 2026-11-13

Summary

PixVid is a method that lifts pretrained latent video diffusion models into pixel space for frame-by-frame generation. Frontier video models operate in a VAE latent space that compresses video spatially and temporally, enabling tractable training but limiting high-resolution, controllable, and interactive generation: doubling resolution quadruples token count, conditions apply to frame groups rather than individual frames, and each response must wait for an entire multi-frame block. PixVid retains the diffusion transformer and replaces the VAE decoder with a convolutional head that renders outputs as pixels. The authors find that predicting clean video instead of velocity, combined with a larger noise shift, is key to quality. Fine-tuned from Wan2.1-14B with only about 1.3 million training clips, PixVid matches latent baselines while enabling 2x higher resolution at the same per-frame token count, maintaining depth-conditioned video editing alignment under fast motion, and responding to actions on the next frame. Paper: arXiv 2611.17336.

Paper Overview

Research Area: Computer Vision (CV) Authors: Yanke Song, Shuo Chen, Chris Rockwell, Guande He, Nan Liu, Desai Xie, Tianpei Gu, Yong-Hyun Park, Aryan V S, Dennis Yuren Menn, Ming-Hsuan Yang, Xun Huang Published: 2026-11-12 arXiv: 2611.17336

Abstract

Frontier video generation models work in a VAE latent space that compresses video in both space and time. This compression made training tractable, but now limits high-resolution, controllable, and interactive generation — doubling resolution quadruples the token count, conditions apply to groups of frames rather than individual ones, and each response must wait for the rest of its multi-frame block. We present PixVid, a simple yet effective method that lifts a pretrained latent video model to directly generate every frame in pixel space. PixVid keeps the diffusion transformer and replaces the VAE decoder with a convolutional head that renders its outputs into pixels. We find that predicting clean video rather than velocity, together with a larger noise shift, is key to quality. Fine-tuned from Wan2.1-14B with only about 1.3 million training clips, PixVid reaches parity with the latent baseline. Working frame-by-frame in pixel space, it generates at 2x higher resolution with the same per-frame token count, preserves alignment for depth-conditioned video editing under fast motion, and responds instantly to each action on the next frame.

Key Results

  • Direct pixel-space, per-frame generation from a pretrained latent video model
  • Convolutional head replaces the VAE decoder; diffusion transformer retained
  • Clean-video prediction + larger noise shift are critical to quality
  • ~1.3M training clips fine-tuned from Wan2.1-14B
  • 2x higher resolution at identical per-frame token count; per-frame conditioning and next-frame interactivity
Paper link: https://arxiv.org/abs/2611.17336

--- *Auto-collected on 2026-11-14*

Tags

#video-generation#diffusion-models#pixvid#cv#arxiv#pixel-space#wan2-1

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637104