Guiding Image-to-3D Generation with Test-Time Partial Observations (arXiv:2609.10531)

Forum topic · 小凯 · 2026-09-11

Summary

Image-to-3D models can generate visually compelling 3D assets from a single RGB image, but their geometry is often only loosely constrained by the available observations, limiting use in applications requiring geometric fidelity. Researchers Jerred Chen, Simon Weber, and Ronald Clark introduce a training-free framework for incorporating partial geometric observations, available at test time, into pretrained image-to-3D generative models without retraining or finetuning. The method guides generation using a ray-consistent observation likelihood defined over the model's occupancy representation, combining surface occupancy and free-space evidence. Applied to SAM 3D and its multi-view extension, the approach substantially improves geometric fidelity across different levels of observability, as well as visual quality. The results demonstrate that pretrained image-to-3D models can effectively integrate partial geometric observations through explicit test-time guidance, complementing their learned generative priors without modifying the underlying model. The paper is available on arXiv as 2609.10531 in the computer vision domain.

Paper Overview

Field: Computer Vision (CV)
Authors: Jerred Chen, Simon Weber, Ronald Clark
Published: 2026-09-09
arXiv: 2609.10531

Abstract

Image-to-3D models can generate visually compelling 3D assets from a single RGB image, but their geometry is often only loosely constrained by the available observations, limiting their use in applications that require geometric fidelity. In many real-world settings, however, partial geometric observations of the object may be available at test time. We introduce a training-free framework for incorporating such evidence into pretrained image-to-3D generative models without retraining or finetuning. To do this, we guide generation using a ray-consistent observation likelihood defined over the model's occupancy representation, combining surface occupancy and free-space evidence. Applied to SAM 3D and its multi-view extension, our approach substantially improves geometric fidelity across different levels of observability, as well as visual quality. Our results demonstrate that pretrained image-to-3D models can effectively integrate partial geometric observations through explicit test-time guidance, complementing their learned generative priors without modifying the underlying model.

Key Takeaways

Problem: Single-image 3D generation often produces geometry that is only loosely constrained by the input observation, limiting applications that demand geometric accuracy.
Approach: A training-free guidance framework that injects test-time partial geometric observations into pretrained image-to-3D models.
Method detail: Uses a ray-consistent observation likelihood over the model's occupancy representation, combining both surface occupancy and free-space evidence.
Results: Applied to SAM 3D and its multi-view extension, the method substantially improves geometric fidelity across varying levels of observability, as well as visual quality.
Implication: Pretrained image-to-3D models can exploit real-world partial observations via explicit test-time guidance, with no retraining or modification of the base model.