Paper Overview
- Research area: Computer Vision
- Authors: Runyi Yang, Deheng Zhang, Xiaoye Wang, Kanzhi Wu, Lei Sun et al.
- arXiv: 2609.26795
- On 50 ScanNet++ scenes, evidence-based selection and registration retry increase matched F1 at 20 mm from 0.336 to 0.383 at fixed retention.
- Further tests demonstrate asset executability, manipulation gains over a single-generator baseline, and the visual cost of conversion.
- Together, these results demonstrate that φ-RIE enables interactive scene conversion.
Original Abstract
3D Gaussian Splatting (3DGS) can reconstruct a captured scene photorealistically, but the resulting representation does not by itself support physical interaction. Robot simulation instead requires object-level change, i.e., objects must move independently, make contact, and reveal previously occluded surroundings. This gap arises because object appearance may remain entangled with the background, while hidden object geometry and occluded background content may be unobserved.To address this challenge, the authors present φ-RIE, a Gaussian-native pipeline that converts selected objects into movable simulator assets while preserving the remaining reconstruction. The key observation is that asset construction and source removal should be coupled: one object identity should define both the movable asset and the scene content to remove and complete. Scene Observation supplies shared evidence to Coupled Scene Construction, which creates registered assets and completed background Gaussians for simulator-driven rendering in an Interactive Environment. This coupling preserves unedited Gaussians while aligning visual and physical state.