Paper Overview
- Field: Computer Vision (CV)
- Authors: Inhee Lee, Sangwon Baik, Sungjoo Kim, Hyeonwoo Kim, Hyunsoo Cha, Hanbyul Joo
- Published: 2026-06-02
- arXiv: 2606.03994
- Single-image lifted scenes often fail in physics simulation: objects interpenetrate, hover, or sink.
- Prior physics-aware approaches only perform post-hoc layout cleanup, ignoring geometric errors.
- SimuScene uses the physics engine diagnostically inside the generative loop, not just for cleanup.
- Correction signals trigger gravity-axis stretching and non-modal shape resampling.
- Achieves state-of-the-art results on physical stability and geometric alignment benchmarks.
Summary
Reconstructing interactive, simulation-ready 3D scenes from a single image is a critical bottleneck for robotic manipulation. While recent single-image lifters recover plausible per-object shapes, composing them yields scenes that collapse under physical simulation due to interpenetrating, hovering, or sinking objects. Existing physics-aware methods address this strictly as a post-hoc layout correction, leaving the underlying geometric errors unresolved.
To address this, the authors introduce SimuScene, a compositional 3D reconstruction pipeline that puts physics in the loop of shape and layout estimation. Rather than using physics merely for layout cleanup, SimuScene utilizes the physics engine as a diagnostic measurement tool during the generative process itself. By diagnostically simulating reconstructed objects under gravity, the pipeline converts penetration and support failures into quantitative correction signals that drive gravity-axis stretching and non-modal shape resampling.
This physics-informed feedback loop mitigates accumulated reconstruction errors and produces stable, simulation-ready compositional 3D scenes. Extensive experiments demonstrate state-of-the-art performance on physical stability and geometric alignment benchmarks.