[论文] WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memo...

论文概要

研究领域: CV 作者: Wangbo Yu, Kunhao Liu, Wenbo Hu, Shenghai Yuan, Chaoran Feng, Haiyang Zhou, Yukun Huang, Yiran Wang, Wang Zhao, Yingmin Luo, Ying Shan 发布时间: 2026-09-21 arXiv: 2609.24984

中文摘要

视频世界模型使动态环境的交互式探索成为可能，但在长时间跨度和多视角下难以保持一致性。我们提出 WorldCrafter，一种学习可查询相机视角的隐式 3D 感知记忆的视频世界模型。核心思路是让请求的视角决定多视角证据如何被压缩到视频生成器有限的 token 预算中。记忆编码器和姿态条件读出模块与视频生成器联合训练，在去噪之前将历史观测整合到一组固定数量的目标视角 token 中，无需显式的基于深度的对应关系。通过将这种记忆与近期时间上下文和少步蒸馏相结合，WorldCrafter 实现了从单张输入图像或文本提示开始的流式场景探索。在静态和动态场景的实验中，该方法在长时间一致性和相机控制精度方面取得了显著提升，同时在分钟级探索中保持了视觉质量。

原文摘要

Video world models enable interactive exploration of dynamic environments, yet struggle to respect prior observations over long horizons and across viewpoints. We present WorldCrafter, a video world model that learns a camera-queryable implicit 3D-aware memory for this purpose. The key insight is to let the requested viewpoint shape how multi-view evidence is compressed into the video generator's limited token budget. Trained jointly with the video generator, a memory encoder and pose-conditioned readout module integrate historical observations into a fixed set of target view-specific tokens before denoising, without explicit depth-based correspondences. By combining this memory with recent temporal context and few-step distillation, WorldCrafter enables streaming scene exploration from a ...

#论文 #arXiv #CV #小凯