Paper Overview
Research Area: CV Authors: Luka Milivojevic, Nikola Popovic, Sayan Deb Sarkar, Sebastian Koch, Iro Armeni, Luc Van Gool, Danda Pani Paudel Published: 2026-09-25 arXiv: 2609.31595
Summary
3D scene graphs provide structured representations of complex environments by encoding objects, their semantic attributes, and the spatial and functional relationships between them. Existing methods for 3D scene graph generation have several fundamental limitations: they depend on complex multi-stage pipelines with explicit intermediate representations, making the systems fragile and prone to error propagation; they assume access to ground-truth object annotations at inference, deviating from real-world scenarios; and they rely on closed-source models that hinder open-source deployment, or suffer from unacceptably slow inference.
The authors propose GraphWrit3R, a simple end-to-end method that takes a 3D point cloud, Gaussian Splats, or a combination of both as input, and directly outputs a complete scene graph (a structured JSON script) listing all objects, their semantic attributes, and their relationships, avoiding all of the above limitations. The multi-modal input design is purely for generality: a single set of weights handles diverse scenes. Point cloud inputs are encoded with Sonata, and Gaussian Splat inputs are encoded with Chorus. Both modalities are projected onto a shared voxel grid and fused via a novel per-voxel contrastive alignment loss, then decoded by a large language model. As a natural byproduct of the LLM, GraphWrit3R also supports open-vocabulary queries.
On the 3DSSG benchmark, the method achieves state-of-the-art recall on object classes, predicates, and triplets, outperforming approaches that rely on ground-truth object annotations at inference. The paper also provides qualitative results and ablations over input modality configurations, contrastive loss formulations, and token fusion strategies.
Original Abstract (excerpt)
> 3D scene graphs provide a structured representation of complex environments by encoding objects, their semantic attributes, and the spatial and functional relationships between them. Current approaches for 3D scene graph generation suffer from several fundamental limitations. They rely on complex multi-stage pipelines with explicit intermediate representations, making systems fragile and prone to error propagation. They assume access to ground-truth object annotations during inference, which deviates from real-world scenarios. They depend on proprietary models, hindering open-source deployment, or incur prohibitively slow inference. We present GraphWrit3R, a simple end-to-end method that takes a 3D point cloud, Gaussian Splats, or a combination of both as input, and directly outputs a comp...
---
*Auto-collected on 2026-09-29*