[论文] Gen4P: Video Generators are 4D Perceivers

论文概要

研究领域: CV
作者: Yitong Xia, Zeren Jiang, Chuanxia Zheng
发布时间: 2026-10-15
arXiv: 2610.22371

中文摘要

我们引入 Gen4P，这是一个从单目视频中恢复 4D 几何和运动的统一框架。我们表明，预训练视频生成器可以在无需迭代去噪的情况下被重新用作前馈 4D 感知器。基于按需查询，Gen4P 在单次编码器通过中处理干净视频潜变量，并通过轻量级交叉注意力解码器预测所需的 4D 属性。按需查询形式化实现了灵活的空间采样，我们提出的连续时间查询进一步提供了时间灵活性。为了增强几何知识，我们通过纳入预训练视频深度 LoRA 作为冻结几何先验，在不增加额外可训练参数的情况下提升整体性能。Gen4P 在 3D 点跟踪、视频深度和相机姿态估计任务上达到了最先进或具有竞争力的性能。在野外视频上的定性结果展示了其在跟踪复杂运动同时保持精细结构方面的能力。

原文摘要

We introduce Gen4P, a unified framework for recovering 4D geometry and motion from monocular video. We show that a pretrained video generator can be repurposed into a feed-forward 4D perceiver without iterative denoising. Building on on-demand querying, Gen4P processes clean video latents in a single encoder pass and predicts the requested 4D attributes through a lightweight cross-attention decoder. The on-demand query formulation enables flexible spatial sampling, and our proposed continuous-time queries further provide temporal flexibility. To strengthen geometric knowledge, we propose a straightforward plug-in solution by incorporating a pretrained video-depth LoRA as a frozen geometry prior, improving overall performance without adding extra trainable parameters. Gen4P achieves state-o...

自动采集于 2026-10-17

#论文 #arXiv #CV #小凯