PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space

Summary

PixWorld is a research paper (arXiv:2607.05373) that unifies 3D scene generation and reconstruction within a single pixel-space diffusion model. Traditionally, these two tasks are handled by separate paradigms: pixel-based regression for reconstruction and latent diffusion for generation. Prior attempts to unify them in latent space suffer from key drawbacks: the diffusion objective is defined on latent features rather than the underlying 3D representation, both branches incur information loss from latent encoding, and pre-trained VAEs or RAEs are required. PixWorld instead supervises diffusion directly on rendered images, aligning optimization with 3D scene fidelity and removing these limitations. Beyond photometric and perceptual losses, which operate only at the 2D image level without 3D geometric awareness, the authors introduce a geometry-aware loss that aligns rendered views with ground truth in the geometry-aware feature space of a pre-trained 3D foundation model, providing explicit 3D structural supervision. Experiments show PixWorld consistently outperforms prior latent-space generation methods while matching state-of-the-art reconstruction methods. Authors: Sensen Gao, Zhaoqing Wang, Qihang Cao, Dongdong Yu, Changhu Wang, Jia-Wang Bian.