[论文] FastFrames: Video Generation at Interactive Latency
论文概要
研究领域: CV
作者: Gene Chou, Guandao Yang, Fujun Luan, Lars Mescheder, Wei Dong, Stephan Richter, Vladlen Koltun
发布时间: 2026-11-17
arXiv: 2611.23226
中文摘要
自回归视频扩散模型基于新观察和输入生成帧,是交互式视觉体验的有前景媒介。然而现有模型速度比流畅交互所需慢几个数量级。本文提出 FastFrames——快速自回归视频扩散模型家族,在不牺牲质量的前提下实现交互延迟。我们对最先进模型的推理流程进行分析,发现通常被视为固定实现细节的操作(如 VAE 解码和 KV 缓存更新)实际占端到端推理延迟一半以上。1.3B 变体在单块 B200 GPU 上以每帧 36 毫秒(28 RPS)运行,质量媲美慢近 8 倍的最先进模型。14B 变体在 8 卡 B200 节点上以每帧 45 毫秒(22 RPS)运行,视觉质量显著提升。FastFrames 在相机条件和物理条件视频生成等应用中实现实时交互。
原文摘要
Autoregressive video diffusion models generate frames based on new observations and inputs, making them a promising medium for interactive visual experiences. However, existing models are orders of magnitude too slow for fluid interaction. This paper introduces FastFrames, a family of fast autoregressive video diffusion models that achieves interactive latency without compromising quality. We profile the inference pipeline of state-of-the-art models and find that operations often treated as fixed implementation details, such as VAE decoding and KV cache updates, in fact account for over half of end-to-end inference latency. We challenge the assumption that these components are indispensable and replace them with lightweight alternatives. Our 1.3B variant runs at 36 ms per frame, or 28 resp...
自动采集于 2026-11-19
#论文 #arXiv #CV #小凯