English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

phi-RIE: Turning 3D Gaussian Splatting Reconstructions into Interactive Simulation Environments

Forum topic · 小凯 · 2026-09-24

Summary

phi-RIE is a Gaussian-native pipeline that converts photorealistic 3D Gaussian Splatting (3DGS) reconstructions into interactive environments suitable for robot simulation. While 3DGS reconstructs captured scenes with high visual fidelity, its representations do not support physical interaction: objects cannot move independently, make contact, or reveal previously occluded surroundings. phi-RIE addresses this by coupling asset construction with source removal, so that a single object identity defines both the movable simulator asset and the scene content to remove and complete. Scene observation supplies shared evidence for coupled scene construction, producing registered assets and completed background Gaussians for simulator-driven rendering. The coupling preserves unedited Gaussians while aligning visual and physical states. On 50 ScanNet++ scenes, evidence-based selection and registration retry raise matched F1 at 20mm from 0.336 to 0.383 at fixed retention, with further tests showing asset executability, manipulation gains over a single-generator baseline, and the visual cost of conversion. Paper by Runyi Yang, Deheng Zhang, Xiaoye Wang, Kanzhi Wu, Lei Sun, et al., arXiv:2609.26795.

Paper Overview

  • Research area: Computer Vision
  • Authors: Runyi Yang, Deheng Zhang, Xiaoye Wang, Kanzhi Wu, Lei Sun et al.
  • arXiv: 2609.26795
  • Original Abstract

    3D Gaussian Splatting (3DGS) can reconstruct a captured scene photorealistically, but the resulting representation does not by itself support physical interaction. Robot simulation instead requires object-level change, i.e., objects must move independently, make contact, and reveal previously occluded surroundings. This gap arises because object appearance may remain entangled with the background, while hidden object geometry and occluded background content may be unobserved.

    To address this challenge, the authors present φ-RIE, a Gaussian-native pipeline that converts selected objects into movable simulator assets while preserving the remaining reconstruction. The key observation is that asset construction and source removal should be coupled: one object identity should define both the movable asset and the scene content to remove and complete. Scene Observation supplies shared evidence to Coupled Scene Construction, which creates registered assets and completed background Gaussians for simulator-driven rendering in an Interactive Environment. This coupling preserves unedited Gaussians while aligning visual and physical state.

    Key Results

  • On 50 ScanNet++ scenes, evidence-based selection and registration retry increase matched F1 at 20 mm from 0.336 to 0.383 at fixed retention.
  • Further tests demonstrate asset executability, manipulation gains over a single-generator baseline, and the visual cost of conversion.
  • Together, these results demonstrate that φ-RIE enables interactive scene conversion.
*Auto-collected on 2026-09-24.*

Tags

#3d-gaussian-splatting#robot-simulation#scene-reconstruction#computer-vision#interactive-environments#arxiv#simulator-assets

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178635138