[论文] FastFrames: Video Generation at Interactive Latency
研究领域: CV 作者: Gene Chou, Guandao Yang, Fujun Luan, Lars Mescheder, Wei Dong, Stephan Richter, Vladlen Koltun 发布时间: 2026-11-17 arXiv: 2611.23226
论文概要
研究领域: CV 作者: Gene Chou, Guandao Yang, Fujun Luan, Lars Mescheder, Wei Dong, Stephan Richter, Vladlen Koltun 发布时间: 2026-11-17 arXiv: 2611.23226
中文摘要
自回归视频扩散模型基于新观察和输入生成帧,是交互式视觉体验的有前景媒介。然而现有模型速度比流畅交互所需慢几个数量级。本文提出 FastFrames——快速自回归视频扩散模型家族,在不牺牲质量的前提下实现交互延迟。我们对最先进模型的推理流程进行分析,发现通常被视为固定实现细节的操作(如 VAE 解码和 KV 缓存更新)实际占端到端推理延迟一半以上。1.3B 变体在单块 B200 GPU 上以每帧 36 毫秒(28 RPS)运行,质量媲美慢近 8 倍的最先进模型。14B 变体在 8 卡 B200 节点上以每帧 45 毫秒(22 RPS)运行,视觉质量显著提升。FastFrames 在相机条件和物理条件视频生成等应用中实现实时交互。
原文摘要
Autoregressive video diffusion models generate frames based on new observations and inputs, making them a promising medium for interactive visual experiences. However, existing models are orders of magnitude too slow for fluid interaction. This paper introduces FastFrames, a family of fast autoregressive video diffusion models that achieves interactive latency without compromising quality. We profile the inference pipeline of state-of-the-art models and find that operations often treated as fixed implementation details, such as VAE decoding and KV cache updates, in fact account for over half of end-to-end inference latency. We challenge the assumption that these components are indispensable and replace them with lightweight alternatives. Our 1.3B variant runs at 36 ms per frame, or 28 resp...
*自动采集于 2026-11-19*
#论文 #arXiv #CV #小凯