[论文] METTA: Memory-Enhanced Test-Time Adaptation for Meta-Reinforcement Lea...

论文概要

研究领域: ML
作者: Ali Najar, Hosein Hasani, Amir Kooshan Fattah, Yasamin Rajabi, Mahdieh Soleymani Baghshah
发布时间: 2026-10-22
arXiv: 2610.31656

中文摘要

元强化学习旨在训练能从有限交互中快速适应未见任务的智能体，但这需要在长交互历史中表征任务证据。经典的循环智能体将经验压缩到隐藏状态中，而基于 Transformer 的智能体以二次方的内存和计算成本从过去轨迹中检索。我们提出 METTA（记忆增强测试时适应），将快速适应阶段分解为两个时间尺度：因果神经记忆以线性序列复杂度执行片段内的细粒度适应，紧凑的片段级摘要支持跨片段的长程任务推断。此外，配备神经记忆为智能体提供了足够的表达能力与测试时训练能力，使其在元测试时能有效适应未见的保留任务和任务族。在未见过的 Meta-World ML10/45 测试环境中，METTA 以较低的训练预算和较短的墙钟时间实现了超过 50% 的成功率。在长上下文稀疏奖励的 PointMaze 任务上，它通过更具结构化的探索行为超越了基于 Transformer 的基线。

原文摘要

Meta-reinforcement learning aims to train agents that adapt quickly to unseen tasks from limited interaction, but this requires representing task evidence over long interaction histories. Classical recurrent agents compress experience into hidden states, while transformer-based agents retrieve from past trajectories at quadratic cost. We propose METTA, which decomposes the fast adaptation phase into two temporal scales: a causal neural memory performs fine-grained within-episode adaptation with linear complexity, while compact episode-level summaries support long-range cross-episode task inference. On unseen Meta-World test environments, METTA achieves a success rate over 50% while maintaining low training budget. On long-context sparse-reward PointMaze tasks, it outperforms transformer-ba...

自动采集于 2026-10-24

#论文 #arXiv #ML #小凯