GraphWrit3R: End-to-End 3D Scene Graph Generation from Point Clouds and Gaussian Splats

Forum topic · 小凯 · 2026-09-29

Summary

GraphWrit3R is an end-to-end method for 3D scene graph generation presented by researchers including Luka Milivojevic, Iro Armeni, Luc Van Gool, and Danda Pani Paudel (arXiv:2609.31595). Unlike existing multi-stage pipelines that rely on ground-truth object annotations or proprietary/slow models, GraphWrit3R takes a 3D point cloud, Gaussian Splats, or both as input and directly outputs a complete scene graph as structured JSON, listing all objects, their semantic attributes, and relationships. Point clouds are encoded with Sonata and Gaussian Splats with Chorus; both modalities are projected onto a shared voxel grid, fused via a novel per-voxel contrastive alignment loss, and decoded by a large language model, which also enables open-vocabulary queries. On the 3DSSG benchmark, the method achieves state-of-the-art recall on object classes, predicates, and triplets, surpassing approaches that use ground-truth object annotations at inference. The paper includes qualitative results and ablations on input modality configurations, contrastive loss formulations, and token fusion strategies.

Paper Overview

Field: Computer Vision Authors: Luka Milivojevic, Nikola Popovic, Sayan Deb Sarkar, Sebastian Koch, Iro Armeni, Luc Van Gool, Danda Pani Paudel Published: 2026-09-25 arXiv: 2609.31595

Summary

3D scene graphs provide a structured representation of complex environments by encoding objects, their semantic attributes, and the spatial and functional relationships between them. Current approaches for 3D scene graph generation suffer from several fundamental limitations:

They rely on complex multi-stage pipelines with explicit intermediate representations, making systems fragile and prone to error propagation.
They assume access to ground-truth object annotations during inference, which deviates from real-world scenarios.
They depend on proprietary models, hindering open-source deployment, or incur prohibitively slow inference.

GraphWrit3R is a simple end-to-end method that takes a 3D point cloud, Gaussian Splats, or a combination of both as input, and directly outputs a complete scene graph (a structured JSON script) listing all objects, their semantic attributes, and their relationships, avoiding all of the above limitations. Supporting multiple input modalities is purely a matter of generality: a single set of weights handles diverse scenes.

Method

Point cloud inputs are encoded with Sonata; Gaussian Splat inputs are encoded with Chorus.
Both modalities are projected onto a shared voxel grid and fused via a novel per-voxel contrastive alignment loss.
A large language model then decodes the fused representation into the scene graph.
As a natural byproduct of using an LLM, GraphWrit3R supports open-vocabulary queries.

Results

On the 3DSSG benchmark, the method achieves state-of-the-art recall on object classes, predicates, and triplets.
It outperforms methods that rely on ground-truth object annotations at inference time.
The paper provides qualitative results and ablation studies on input modality configurations, contrastive loss formulations, and token fusion strategies.

---

*Auto-collected on 2026-09-29.*

GraphWrit3R: End-to-End 3D Scene Graph Generation from Point Clouds and Gaussian Splats

Summary

Paper Overview

Summary

Method

Results

Tags