[论文] GenIA: Generative Reconstruction with Test-Time Input Alignment

论文概要

研究领域: CV
作者: Stefano Esposito, Naama Pearl, Polina Karpikova, Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder, Andreas Geiger, Jonathon Luiten
发布时间: 2026-10-08
arXiv: 2610.12388

中文摘要

从单目或稀疏多视角观测重建完整的 3D 物体资产仍具挑战。生成式 3D 基础模型能补全观测视角之外的几何，但其预测未必忠实复现观测到的几何、外观或姿态。我们提出 GenIA——一个测试时输入对齐生成框架，无需重训基础模型即可将 SAM3D 的生成先验接地到几何与光度观测。我们通过从几何推导平移和尺度来改进物体姿态（同时保留学习到的旋转先验），并通过可见性偏置注意力、跨观测融合以及去噪过程中的可微渲染引导来对齐外观。可选的去噪后细化进一步适配外观隐变量、轻量化解码器适配器和物体放置。框架还支持外部提供的几何：给定动态物体的时间形状时，能恢复共享的、输入对齐的规范外观和稳定的世界空间放置。在合成和真实基准上，GenIA 在单目、多视角和动态输入的物体姿态预测与重建上均优于近期基于优化的逐帧图像转 3D 和视频转 4D 方法。

原文摘要

Reconstructing complete 3D object assets from monocular or sparse multi-view observations remains challenging. Generative 3D foundation models can complete object geometry beyond the observed views, but their predictions may not faithfully reproduce the observed geometry, appearance, or pose. We introduce GenIA, a framework for test-time input-aligned generation that grounds SAM3D's generative prior in geometric and photometric observations without retraining the foundation model. We improve object pose by deriving translation and scale from geometry while retaining the learned rotation prior, and align appearance through visibility-biased attention, cross-observation fusion, and differentiable rendering guidance during denoising. An optional post-denoising refinement further adapts the ap...

自动采集于 2026-10-12

#论文 #arXiv #CV #小凯