Paper Overview
Field: Computer Vision Authors: Song Tang, Shuming Hu, Xincheng Shuai, Henghui Ding, Yu-Gang Jiang Published: 2026-07-02 arXiv: 2607.02497 Category: cs.CV
Summary
Existing referring segmentation models passively process static images captured from fixed perspectives, limiting their applicability in Embodied AI, where agents must perform active perception in continuous 360° environments.
To bridge this gap, the authors introduce a novel task: Active Panoramic Referring Segmentation (APRS). In this setting, an agent must adjust its viewing direction (Δθ, Δφ) to explore the 360° environment and seek out the object specified by a user instruction for segmentation.
PanoSeeker
To tackle this challenging task, the authors propose PanoSeeker, a memory-augmented agent for efficient APRS. Rather than relying on heuristic scanning, PanoSeeker integrates a Vision-Language Model (VLM) with EgoSphere, an explicit spatial visual memory. By progressively integrating sequential local observations into a unified 360° representation, EgoSphere enables the agent to plan efficient, non-redundant search trajectories. Once the target is found, the agent performs active viewpoint alignment and outputs the segmentation mask.
Training Data and Method
The team also curated an expert-annotated dataset of search trajectories with memory timelines, used for supervised fine-tuning. This is followed by reinforcement learning post-training to explicitly optimize PanoSeeker's exploration efficiency.
Results
Extensive experiments on the newly established APRS benchmark demonstrate that PanoSeeker achieves excellent performance in both search efficiency and segmentation accuracy, significantly outperforming adapted state-of-the-art baseline methods.
---
*Automatically collected on 2026-07-06.*