[论文] Workspace Models: Lightweight Robotic Memory via Saliency-Driven Super...

论文概要

研究领域: ML 作者: Nitish Dashora, Douglas Chen, Idan Shenfeld, John Marangola, Pulkit Agrawal, Max Simchowitz 发布时间: 2026-09-17 arXiv: 2609.20820

中文摘要

复杂的机器人操控任务通常需要对过去事件和动作的长期记忆。由于以完整历史为条件会使策略容易产生虚假相关性并降低性能，许多策略记忆方法通过在推理回路中进行昂贵的 VLM 查询来压缩历史信息，仅处理与任务相关的信息。本文提出了一种替代方案：在训练时进行计算密集的 VLM 查询，学习一种轻量级潜在记忆，可在部署时高效查询。我们将这种表示称为"工作空间令牌"（workspace token），其训练过程为：(1) 使用 VLM 识别完成任务所需的当前和历史信息，(2) 通过集合重建解码器损失将这些信息蒸馏到工作空间令牌中。在仿真和硬件实验中，我们表明工作空间令牌可在部署时直接替代观察值，使策略无需 VLM 推理回路即可解决记忆密集型任务。有趣的是，我们发现工作空间令牌不仅更轻量，而且还能带来更好的策略性能。

原文摘要

Complex robotic manipulation tasks frequently require a long-term memory of past events and actions. As conditioning on full histories renders policies prone to spurious correlations and degrades performance, many approaches to policy memory involve compressing historical information through expensive VLM queries in-the-loop to process only task-salient information. In this paper, we propose an alternative approach in which computationally intensive VLM queries are made during train-time to learn a lightweight latent memory that can be efficiently queried at deployment time. Our representation, which we call the workspace token, is trained by (1) using a VLM to identify current and historical information necessary for completing a task, then (2) distilling these into the workspace token us...

#论文 #arXiv #ML #小凯