[论文] PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation

论文概要

研究领域: CV 作者: Haofei Xu, Rundi Wu, Philipp Henzler 发布时间: 2026-07-04 arXiv: 2507.00483

中文摘要

最先进的单图像3D重建方法通常依赖于复杂的混合架构和损失函数，或者将几何压缩到潜在空间以利用预训练的潜在扩散模型。在这项工作中，我们证明了这种架构开销和复杂的损失公式是不必要的。我们引入了一个极简的像素空间扩散Transformer，基于普通ViT构建，直接在原始3D点图块上操作，并以预训练DINOv3的图像token为条件。与现有的潜在扩散方法不同，我们从头开始训练扩散骨干网络，消除了对点图token化器的需求。尽管简单，我们的方法超越了复杂的基于潜在的扩散模型，同时比混合替代方案显著更简单。值得注意的是，它产生了更锐利的几何结构，并在高度模糊的区域（如透明物体）中更具鲁棒性。

原文摘要

State-of-the-art single-image 3D reconstruction methods often rely on complex hybrid architectures and loss functions, or compress geometry into latent spaces in order to leverage pre-trained latent diffusion models. In this work, we show that such architectural overhead and intricate loss formulations are unnecessary. We introduce a minimalist pixel-space Diffusion Transformer, built on a plain ViT, that operates directly on raw 3D point map patches and is conditioned on image tokens from a pre-trained DINOv3. Unlike existing latent diffusion approaches, we train our diffusion backbone entirely from scratch, eliminating the need for point map tokenizers. Despite its simplicity, our approach surpasses complex latent-based diffusion models while remaining significantly simpler than hybrid a...

#论文 #arXiv #CV #小凯