Building Rome from a Single Image: Single-Image 3D Scene Generation for Indoor and Outdoor Scenes

Forum topic · 小凯 · 2026-10-08

Summary

This paper, titled "Building Rome from a Single Image" (arXiv:2610.08790), addresses single-image 3D scene generation: producing a complete scene mesh, including surfaces the camera never observed. Pretrained 3D object generators like Trellis 2 encode strong shape priors but are designed for isolated objects in a fixed canonical volume and mostly indoor scenes, since outdoor 3D data is scarce. The authors redesign the object-centric generator to handle both indoor and outdoor scenes while retaining its prior. Key techniques: (a) adaptive scene chunking, where chunk size scales with distance to the camera—small nearby chunks preserve fine detail while large chunks cover distant structures like buildings; (b) lifting image features to capture explicit 2D-3D correspondences and making the model aware of free space, observed surfaces, and unobserved regions; (c) synthesizing roughly 4,000 outdoor scenes to expand training data. Experiments on Tanks and Temples, ScanNet++, and real-world images show the method outperforms all baselines in both geometric accuracy and perceptual quality for indoor and outdoor scenes.

Paper Overview

Research Area: Computer Vision Authors: Jiraphon Yenphraphai, Fang Li, Tianshuo Xu, Depu Meng, Quentin Herau, Yihan Hu, Raymond A. Yeh, Wei Zhan Published: 2026-10-06 arXiv: 2610.08790

Abstract

Single-image scene generation aims to produce a complete 3D scene mesh from a single image, including surfaces the camera did not observe. While pretrained 3D object generators encode a strong shape prior, they are mainly designed for isolated objects in a fixed canonical volume and focus mostly on indoor scenes, since diverse 3D data for outdoor scenes are quite limited.

In this work, the authors present a method that redesigns such an object-centric generator (e.g., Trellis 2) to work on both indoor and outdoor scenes while retaining its prior. They accomplish this by:

(a) Adaptive scene chunking: partitioning the scene into chunks whose size scales relative to the distance to the camera. Nearby chunks have a smaller size to keep finer detail, while distant structures (e.g., buildings) are covered by large chunks.
(b) Explicit 2D-3D correspondence: lifting image features so the generator captures explicit 2D-3D correspondences, and making the model aware of free space, observed surfaces, and unobserved regions.
(c) Synthetic outdoor data: synthesizing approximately 4,000 outdoor scenes to expand training data.

Results

Experiments on Tanks and Temples, ScanNet++, and real-world images show that the method outperforms all baselines in both geometric accuracy and perceptual quality for indoor and outdoor scenes.