[论文] K2V: Generative Keyframing For Physically Plausible Video Generation

论文概要

研究领域: CV 作者: Felix O'Mahony, Daniel Olmeda Reino, Roberto Cipolla, Ayush Tewari 发布时间: 2026-11-12 arXiv: 2611.17324

中文摘要

视频扩散模型已能生成外观与纹理俱佳的样本，但已知存在显著的物理错误。原因在于经典视频生成设定中，所有物理推理必须完全在扩散过程内部同时进行。我们提出将物理推理任务重新分配给一个由语言模型支撑的关键帧生成模块——它允许用语言进行显式推理，并对视频中关键时刻的外观实现细粒度控制。随后以这些关键帧为条件驱动视频生成模型，将扩散模型的角色简化为在物理正确的中间状态之间插值，而非用扩散推断整条序列。结果表明，该方法显著优于现有方法，在 Physics-IQ Verified 基准和 PhyGenBench 上均达到最优。

原文摘要

Video diffusion models are able to generate samples with high-quality appearance and textural qualities. However, content produced by these methods is known to exhibit significant physical errors. This is because, in a classical video generation setting, all physical reasoning must take place simultaneously and exclusively within the diffusion process.We propose to resolve these issues by redistributing the task of physical reasoning onto a language model-backed keyframe generation module, which allows for explicit reasoning in language, and fine-grained control over the appearance of the video at important moments in the video. We then condition a video generation model with these keyframes, reducing the role of the diffusion model to the simpler task of interpolating between physically-c...

#论文 #arXiv #CV #小凯