Align4D: Alignment Is All You Need For X-to-4D Generation

Forum topic · 小凯 · 2026-07-04

Summary

Align4D is a flexible framework for arbitrary modality-to-4D (X-to-4D) generation, presented in arXiv paper 2507.00484 by Qiaowei Miao, Kehan Li, and Yawei Luo. The method converts any user-defined modality input into coherent video-3D pairs, using video to guide 4D motion and 3D data to shape 4D geometry. It introduces three key techniques: (1) Object Distance Alignment, which searches Video-Aligned and Multiview-Aligned Object Distances (VAOD/MAOD) to reconcile 4D renderings with video and multiview diffusion model priors; (2) Motion-Geometry Joint Alignment, which constrains known and unknown views via synchronized video and 3D inputs for consistent 4D generation; and (3) Asynchronous Optimization, decoupling Gaussian attribute and deformation network training to enhance motion and geometry fidelity. The authors also propose the X4D dataset integrating prompts, images, videos, and 3D data for benchmarking. Experiments on X4D and Consistent4D show state-of-the-art quality and consistency in X-to-4D generation.

Paper Overview

Research Area: Computer Vision (CV) Authors: Qiaowei Miao, Kehan Li, Yawei Luo Published: 2026-07-04 arXiv: 2507.00484

Abstract

Generative diffusion models excel at synthesizing high-quality images, videos, and 3D content under multimodal control. However, arbitrary user-defined modality-to-4D (X-to-4D) generation remains challenging due to the high cost of constructing diverse datasets and the limited scalability of existing methods.

This paper presents Align4D, a flexible framework that translates any-modal input into coherent video-3D pairs, using video to guide 4D motion and 3D data to shape 4D geometry.

Key Techniques

1. Object Distance Alignment — searches Video-Aligned and Multiview-Aligned Object Distances (VAOD/MAOD) to reconcile 4D renderings with video and the priors of multiview diffusion models. 2. Motion-Geometry Joint Alignment — constrains known and unknown views via synchronized video and 3D inputs, ensuring consistent 4D generation. 3. Asynchronous Optimization — decouples Gaussian attribute and deformation network training to enhance motion and geometry fidelity.

Dataset & Results

The authors further propose the X4D dataset, integrating prompts, images, videos, and 3D data for benchmarking. Experiments on X4D and Consistent4D demonstrate that Align4D achieves state-of-the-art quality and consistency in X-to-4D generation.

--- *Auto-collected on 2026-07-04*