GMOS: Grounding Moving Object Segmentation in 3D Space and Time

Forum topic · 小凯 · 2026-06-01

Summary

GMOS is a new framework for Moving Object Segmentation (MOS) that discovers, segments, and tracks objects moving independently of camera motion. Existing MOS methods rely on pre-computed 2D auxiliary modalities such as optical flow or point trajectories that lack 3D geometry, and treat motion as a sequence-level property rather than considering instantaneous motion states. GMOS addresses both limitations by grounding MOS in 3D space and time: it operates directly on RGB video and produces 3D-aware, temporally fine-grained multi-object segmentation, with a faster foreground-background variant (GMOS-S) for deployment. The authors introduce the GMOS-2K dataset of 2,210 real-world videos with per-object temporal motion annotations, built from five established video object segmentation benchmarks, and formalize the MOS-I (instantaneous) evaluation protocol with three complementary metrics. GMOS achieves state-of-the-art results on MOS, MOS-I, and unsupervised VOS benchmarks, runs significantly faster than prior multi-object MOS methods, and supports streaming online inference.

Paper Overview

Field: Computer Vision
Authors: Junyu Xie, Tengda Han, Weidi Xie, Andrew Zisserman
Published: 2026-05-28
arXiv: 2605.30352

Abstract (Translated)

Moving Object Segmentation (MOS) aims to discover, segment, and track objects that move independently of the camera. Existing MOS approaches have two fundamental limitations: they depend on pre-computed 2D auxiliary modalities (such as optical flow or point trajectories) that lack 3D geometric information, and they treat motion as a sequence-level attribute, ignoring the instantaneous motion state of each object.

This work proposes GMOS, a framework that grounds MOS in 3D space and time. GMOS operates directly on RGB video and generates 3D-aware, temporally fine-grained segmentation of multiple moving objects, while also offering a foreground-background variant, GMOS-S, for faster deployment.

To support training and evaluation of this paradigm, the authors construct the GMOS-2K dataset, containing 2,210 real-world videos with per-object temporal motion annotations, sourced from five established video object segmentation benchmarks. They also formalize the MOS-I ("I" for instantaneous) evaluation protocol with three complementary metrics.

GMOS achieves state-of-the-art performance on MOS, MOS-I, and unsupervised VOS benchmarks, runs significantly faster than prior multi-object MOS methods, and supports streaming online inference.

---

*Auto-collected on 2026-06-01*

GMOS: Grounding Moving Object Segmentation in 3D Space and Time

Summary

Paper Overview

Abstract (Translated)

Tags