Guiding Image-to-3D Generation with Test-Time Partial Observations (arXiv:2609.10531)

Forum topic · 小凯 · 2026-09-11

Summary

Image-to-3D models can generate visually compelling 3D assets from a single RGB image, but their geometry is often only loosely constrained by available observations. This paper by Jerred Chen, Simon Weber, and Ronald Clark (arXiv:2609.10531) introduces a training-free framework for incorporating partial geometric observations available at test time into pretrained image-to-3D generative models, without retraining or finetuning. The method guides generation using a ray-consistent observation likelihood defined over the model's occupancy representation, combining surface occupancy and free-space evidence. Applied to SAM 3D and its multi-view extension, the approach substantially improves geometric fidelity across different levels of observability, as well as visual quality. The results demonstrate that pretrained image-to-3D models can effectively integrate partial geometric observations through explicit test-time guidance, complementing learned generative priors without modifying the underlying model.

Overview

Research area: Computer Vision Authors: Jerred Chen, Simon Weber, Ronald Clark Published: 2026-09-09 arXiv: 2609.10531

Abstract

Image-to-3D models can generate visually compelling 3D assets from a single RGB image, but their geometry is often only loosely constrained by the available observations, limiting their use in applications that require geometric fidelity. In many real-world settings, however, partial geometric observations of the object may be available at test time. We introduce a training-free framework for incorporating such evidence into pretrained image-to-3D generative models without retraining or finetuning. To do this, we guide generation using a ray-consistent observation likelihood defined over the model's occupancy representation, combining surface occupancy and free-space evidence. Applied to SAM 3D and its multi-view extension, our approach substantially improves geometric fidelity across different levels of observability, as well as visual quality. Our results demonstrate that pretrained image-to-3D models can effectively integrate partial geometric observations through explicit test-time guidance, complementing their learned generative priors without modifying the underlying model.

Key Contributions

Training-free guidance framework: Incorporates partial geometric observations into pretrained image-to-3D generative models at test time, requiring no retraining or finetuning of the underlying model.
Ray-consistent observation likelihood: Guides generation with a likelihood defined over the model's occupancy representation, combining both surface occupancy and free-space evidence.
Strong empirical results: When applied to SAM 3D and its multi-view extension, the method substantially improves geometric fidelity across varying levels of observability, along with visual quality.

Conclusion

Pretrained image-to-3D models can effectively integrate partial geometric observations through explicit test-time guidance, complementing their learned generative priors without modifying the underlying model.

--- *Paper link: arXiv:2609.10531*