METTA: Memory-Enhanced Test-Time Adaptation for Meta-Reinforcement Learning

Forum topic · 小凯 · 2026-10-24

Summary

METTA (Memory-Enhanced Test-Time Adaptation) is a meta-reinforcement learning approach by Najar, Hasani, and colleagues that improves fast adaptation to unseen tasks. Classical recurrent agents compress experience into hidden states, while transformer-based agents retrieve from past trajectories at quadratic memory and compute cost. METTA decomposes the fast adaptation phase into two temporal scales: a causal neural memory performs fine-grained within-episode adaptation with linear sequence complexity, while compact episode-level summaries support long-range cross-episode task inference. The neural memory also gives the agent the expressive capacity for test-time training, enabling effective adaptation to unseen held-out tasks and task families during meta-testing. On unseen Meta-World ML10/45 test environments, METTA achieves a success rate above 50% with a lower training budget and shorter wall-clock time. On long-context sparse-reward PointMaze tasks, it surpasses transformer-based baselines through more structured exploration behavior. The paper is available on arXiv as 2610.31656.

Paper Overview

Field: Machine Learning Authors: Ali Najar, Hosein Hasani, Amir Kooshan Fattah, Yasamin Rajabi, Mahdieh Soleymani Baghshah Published: 2026-10-22 arXiv: 2610.31656

Abstract (translated from Chinese summary)

Meta-reinforcement learning aims to train agents that adapt quickly to unseen tasks from limited interaction, but this requires representing task evidence over long interaction histories. Classical recurrent agents compress experience into hidden states, while transformer-based agents retrieve from past trajectories at quadratic memory and computational cost.

The authors propose METTA (Memory-Enhanced Test-Time Adaptation), which decomposes the fast adaptation phase into two temporal scales:

A causal neural memory performs fine-grained within-episode adaptation with linear sequence complexity.
Compact episode-level summaries support long-range cross-episode task inference.

Additionally, equipping the agent with neural memory provides sufficient expressive capacity and test-time training ability, allowing it to effectively adapt to unseen held-out tasks and task families during meta-testing.

Results

On unseen Meta-World ML10/45 test environments, METTA achieves a success rate over 50% with a lower training budget and shorter wall-clock time.
On long-context sparse-reward PointMaze tasks, it outperforms transformer-based baselines through more structured exploration behavior.

Original Abstract (excerpt)

> Meta-reinforcement learning aims to train agents that adapt quickly to unseen tasks from limited interaction, but this requires representing task evidence over long interaction histories. Classical recurrent agents compress experience into hidden states, while transformer-based agents retrieve from past trajectories at quadratic cost. We propose METTA, which decomposes the fast adaptation phase into two temporal scales: a causal neural memory performs fine-grained within-episode adaptation with linear complexity, while compact episode-level summaries support long-range cross-episode task inference. On unseen Meta-World test environments, METTA achieves a success rate over 50% while maintaining low training budget. On long-context sparse-reward PointMaze tasks, it outperforms transformer-ba...

--- *Auto-collected on 2026-10-24.*