EvoArena: Benchmarking LLM Agent Memory Evolution in Dynamic Environments

Paper Overview

• Field: NLP
• Authors: Jundong Xu, Qingchuan Li, Jiaying Wu, Yihuai Lan, Shuyue Stella Li, Huichi Zhou, Bowen Jiang, Lei Wang, Jun Wang, Anh Tuan Luu, Caiming Xiong, Hae Won Park, Bryan Hooi, Zhiyuan Hu
• Posted: 2026-06-11
• arXiv: 2606.13681

Summary

Large language model (LLM) agents have achieved strong performance on a wide range of benchmarks, yet most evaluations assume static environments. In contrast, real-world deployment is inherently dynamic, requiring agents to continually align their knowledge, skills, and behavior with changing environments and updated task conditions.

To address this gap, the authors introduce EvoArena, a benchmark suite that models environment changes as sequences of progressive updates across terminal, software, and social domains. They further propose EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental evolution through changes in their memory.

Key Findings

• Current agents struggle on EvoArena, achieving an average accuracy of only 39.6% across the evolving terminal, software, and social preference domains.
• EvoMem consistently improves performance, delivering an average gain of 1.5% on EvoArena, while also improving standard benchmarks: 6.1% on GAIA and 4.8% on LoCoMo.
• On EvoArena's chained accuracy — where success requires completing consecutive related evolution subtasks — EvoMem yields a 3.7% improvement.
• Mechanistic analysis shows that EvoMem improves document capture in memory, indicating better preservation of complete evolving environment states.

The results underscore the importance of modeling evolution in both evaluation and memory design for reliable agent deployment.

Original Abstract (excerpt)

> Large language model (LLM) agents have achieved strong performance on a wide range of benchmarks, yet most evaluations assume static environments. In contrast, real-world deployment is inherently dynamic, requiring agents to continually align their knowledge, skills, and behavior with changing environments and updated task conditions. To address this gap, we introduce EvoArena, a benchmark suite that models environment changes as sequences of progressive updates across terminal, software, and social domains. We further propose EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental evolution through changes in their memory...

--- *Auto-collected on 2026-06-15*