[论文] Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video G...

论文概要

研究领域: ML 作者: Haocheng Xi, Yiming Xie, Hexu Zhao, Yiwen Zhang, Michael Liu, Thomas Creavin, Kurt Keutzer, Xiuyu Li, Zhaoyang Lv, Chenfeng Xu, Haiwen Feng 发布时间: 2026-09-17 arXiv: 2609.20744

中文摘要

视频扩散模型在去噪过程中反复处理超长的时空 token 序列，使注意力成为主要计算瓶颈。线性注意力是有吸引力的替代方案，已在近期大语言模型中被广泛采用，但直接应用于视频模型往往无法维持高质量生成所需的细粒度交互。我们提出 Video DeltaNet（VDN），将局部 Softmax 注意力与双向线性记忆结合，以承载长程视频上下文。其线性分支引入 Video Delta Attention（VDA）：每帧联合纳入其空间 token 只更新一次记忆。独立的输出投影与可学习门控校准两个分支，分阶段教师对齐配方逐步将新通路引入预训练模型。我们在 MiniMax H3 上实例化 VDN，对视频-视频交互应用混合注意力，涉及文本或音频的交互仍保留 Softmax。结合八步蒸馏与优化的 SGLang 服务栈，VDN-H3 在八块 NVIDIA B200 GPU 上仅用 6.70 秒即可完成 14.3 秒、768p 视频的 DiT 去噪，相比同卡数的 50 步稠密 H3 基线加速 14.5 倍。

原文摘要

Video diffusion models repeatedly process long spatiotemporal token sequences during denoising, making attention a major computational bottleneck. Linear attention offers an appealing alternative and has been widely adopted in recent large language models, but directly applying it to video models often fails to preserve the fine-grained interactions required for high-quality generation. We present Video DeltaNet (VDN), which combines local Softmax attention with bidirectional linear memory for long-range video context. Its linear branch introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens. Separate output projections and learnable gates calibrate the two branches, while a staged teacher-alignment recipe progressively introd...

#论文 #arXiv #ML #小凯