[论文] MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-La...

论文概要

研究领域: CV
作者: Hao Shi, Weiye Li, Bin Xie
发布时间: 2025-06-06
arXiv: 2506.04876

中文摘要

时间建模对机器人操作至关重要，因为有效的控制既需要对过去交互的记忆，也需要对未来状态的想象。然而，大多数VLA模型主要依赖当前观测，因此在长程时间依赖任务上表现不佳。认知科学表明，人类依赖工作记忆缓存短期上下文、海马系统保存过去经验的episodic记忆、以及内部模型想象未来状态演化。受这些机制启发，我们提出了MemoryVLA++，一个完整的时间建模框架，为VLA模型配备记忆和想象能力。预训练VLM将当前观测编码为感知和认知token，形成工作记忆。这些token查询感知-认知记忆库以检索相关历史上下文，该库存储过去交互的低层细节和高层语义，并通过冗余感知整合进行更新。世界模型在降噪潜在空间中想象未来状态，想象的潜在变量在记忆指导下整合为完整的时间感知token。这些token条件化扩散动作专家以预测时间一致的动作序列。在5个模拟基准和3类真实机器人任务上，我们的方法在通用操作、记忆依赖和想象依赖任务上分别取得+9%、+26%、+28%的提升。

原文摘要

Temporal modeling is essential for robotic manipulation, as effective control requires both memory of past interactions and imagination of future states. However, most VLA models rely primarily on the current observation and therefore struggle with long-horizon, temporally dependent tasks. Cognitive science suggests that humans rely on working memory to buffer short-lived context, the hippocampal system to preserve episodic memory of past experience, and internal models to imagine possible future state evolution. Inspired by these mechanisms, we propose MemoryVLA++, a full temporal modeling framework that equips VLA models with memory and imagination for robotic manipulation. A pretrained VLM encodes the current observation into perceptual and cognitive tokens, forming working memory. Thes...

自动采集于 2026-06-10

#论文 #arXiv #CV #小凯