Overview
Field: Computer Vision (CV) Authors: Shufan Sun, Chen Wang, Enxin Song, Jiatao Gu, Lingjie Liu Published: 2026-09-22 arXiv: 2609.26793
Abstract (Original)
Compositional 3D scene reconstruction has recently been explored from two directions: agentic reasoning that provides semantic understanding of spatial relationships but lacks precise alignment with input images; and visual geometry foundation models that predict dense point maps from input images but the reconstruction quality is limited. Therefore, recovering a complete 3D scene from a single monocular image with accurate inter-object relationships and high-fidelity reconstruction quality remains challenging.
In this paper, we present HARMONY, a hierarchical chain-of-thought framework that leverages both agentic reasoning and visual geometry foundation. Given an image of an indoor scene, starting from an empty 3D floorplan, HARMONY first calibrates the camera against the reference image to establish a semantically-grounded spatial frame, then uses agentic VLM reasoning to recover the 3D room layout and an initial placement order. It then places the objects in a hierarchical order, from wall-mounted elements, free-standing furniture, to dependent decorations on top of furniture.
We also use depth-first traversal for furniture so each placement conditions on previously resolved structure and a reflective feedback loop to avoid error accumulation. After each object placement by VLM, we use the point cloud estimations to perform geometry-based refinement so that the rendered image aligns better with the input.
HARMONY can produce 3D scenes that are semantically consistent and perceptually aligned with the reference image, extending single-image compositional reconstruction to complex indoor scene images. Experiments on synthetic and real-world images demonstrate that HARMONY outperforms the evaluated reconstruction baselines, while qualitative comparisons with GPT-6 Astra suggest more faithful object arrangements and better preservation of scene details.
Key Points (from Chinese Summary)
- Motivation: Agentic reasoning captures spatial relationships semantically but lacks precise alignment with input images; visual geometry foundation models predict dense point maps but with limited reconstruction quality.
- Pipeline: Calibrate camera against the reference image → establish a semantically-grounded spatial coordinate frame → agentic VLM reasoning recovers the 3D room layout and initial placement order.
- Hierarchical placement: Wall-mounted elements first, then free-standing furniture, then dependent decorations on top of furniture. Furniture uses depth-first traversal; each placement conditions on previously established structure.
- Error control: A reflective feedback loop avoids error accumulation across placements.
- Geometric refinement: After each VLM placement, point cloud estimation refines geometry so the rendered image better matches the input.
- Results: Semantically consistent, perceptually aligned 3D scenes on both synthetic and real images; outperforms evaluated baselines, with more faithful object arrangements than GPT-6 Astra in qualitative comparisons.
*Auto-collected on 2026-09-24.*