[论文] Seek to Segment: Active Perception for Panoramic Referring Segmentatio...

论文概要

研究领域: CV
作者: Song Tang, Shuming Hu, Xincheng Shuai
发布时间: 2026-07-04
arXiv: 2507.03235

中文摘要

现有的指代分割（referring segmentation）模型被动处理从固定视角捕获的静态图像，这限制了其在具身智能（Embodied AI）中的应用——智能体必须在连续的 360° 环境中执行主动感知（active perception）。为弥合这一差距，本文引入一个新任务：主动全景指代分割（Active Panoramic Referring Segmentation, APRS）。在此设定下，智能体需要调整其观察方向（Δθ, Δφ）以探索 360° 环境，寻找用户指令指定的目标对象并进行分割。

为应对这一挑战性任务，本文提出 PanoSeeker，一种带记忆增强的 APRS 智能体。与依赖启发式扫描不同，PanoSeeker 将视觉-语言模型（VLM）与 EgoSphere（一种显式空间视觉记忆）相结合。通过将序列化的局部观测逐步整合为统一的全景表示，EgoSphere 使智能体能够规划高效且无冗余的搜索轨迹。一旦找到目标，智能体执行主动视点对齐并输出分割掩码。

此外，本文整理了一个带记忆时间线的专家标注搜索轨迹数据集用于监督微调（SFT），随后通过强化学习后训练来显式优化 PanoSeeker 的探索效率。在新建立的 APRS 基准上的广泛实验表明，PanoSeeker 在搜索效率和分割精度上均显著优于适配后的最先进基线方法。

原文摘要

Existing referring segmentation models passively process static images captured from fixed perspectives, limiting their applicability in Embodied AI, where agents must perform active perception in the continuous 360° environments. To bridge this gap, we introduce a novel task: Active Panoramic Referring Segmentation (APRS). In this setting, an agent is required to adjust its viewing direction (Δθ, Δφ) to explore the 360° environment, seeking the object specified by a user instruction for segmentation. To tackle this challenging task, we propose PanoSeeker, a memory-augmented agent for efficient APRS. Rather than relying on heuristic scanning, PanoSeeker integrates a Vision-Language Model (VLM) with EgoSphere, an explicit spatial visual memory. By progressively integrating sequential local observations into a unified 360° representation, EgoSphere enables the agent to plan efficient and non-redundant search trajectories. Once the target is found, the agent performs active viewpoint alig...

自动采集于 2026-07-05

#论文 #arXiv #CV #小凯