[论文] Alignment Is All You Need For X-to-4D Generation

论文概要

研究领域: CV 作者: Qiaowei Miao, Kehan Li, Yawei Luo 发布时间: 2026-07-04 arXiv: 2507.00484

中文摘要

生成式扩散模型擅长在多模态控制下合成高质量图像、视频和3D内容。然而，由于构建多样化数据集成本高昂且现有方法可扩展性有限，任意用户定义的模态到4D（X-to-4D）生成仍然具有挑战性。本文提出了Align4D，一个灵活的框架，将任意模态输入转化为连贯的视频-3D对，利用视频引导4D运动，利用3D数据塑造4D几何。Align4D引入了三项关键技术：(1) 物体距离对齐，分别搜索视频对齐和多视图对齐的物体距离（VAOD/MAOD），使4D渲染与视频及多视图扩散模型的先验协调一致；(2) 运动-几何联合对齐，通过同步视频和3D输入约束已知和未知视图，确保一致的4D生成；(3) 异步优化，解耦高斯属性和变形网络训练以增强运动和几何保真度。我们进一步提出了X4D数据集，整合提示、图像、视频和3D数据用于基准测试。在X4D和Consistent4D上的实验表明，Align4D在X-to-4D生成中实现了最先进的质量和一致性。

原文摘要

Generative diffusion models excel at synthesizing high-quality images, videos, and 3D content under multimodal control. However, arbitrary user-defined modality-to-4D (X-to-4D) generation remains challenging due to the high cost of constructing diverse datasets and the limited scalability of existing methods. This paper presents Align4D, a flexible framework that translates any-modal input into coherent video-3D pairs, using video to guide 4D motion and 3D data to shape 4D geometry. Align4D introduces three key techniques: (1) Object Distance Alignment, which searches Video-Aligned and Multiview-Aligned Object Distances (VAOD/MAOD), respectively, to reconcile 4D renderings with video and the priors of multiview diffusion models; (2) Motion-Geometry Joint Alignment, which constrains known a...

--- *自动采集于 2026-07-04*

#论文 #arXiv #CV #小凯