EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments

Forum topic · 小凯 · 2026-06-15

Summary

A forum post introduces EvoArena (arXiv:2606.13681), a benchmark suite that evaluates LLM agents in dynamic environments where changes unfold as sequences of progressive updates across terminal, software, and social domains. Unlike most evaluations that assume static conditions, EvoArena requires agents to continually align knowledge, skills, and behavior with evolving task conditions. The paper also proposes EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, letting agents reason about environmental change through memory diffs. Experiments show current agents struggle on EvoArena, averaging only 39.6% accuracy across the evolving domains. EvoMem consistently improves performance, delivering a 1.5% average gain on EvoArena, plus 6.1% on GAIA and 4.8% on LoCoMo standard benchmarks, and a 3.7% gain on chained accuracy where success requires completing consecutive related evolution subtasks. Mechanistic analysis indicates EvoMem improves document capture in memory, better preserving complete evolving environment states.

Paper Overview

Field: NLP Authors: Jundong Xu, Qingchuan Li, Jiaying Wu, Yihuai Lan, Shuyue Stella Li, Huichi Zhou, Bowen Jiang, Lei Wang, Jun Wang, Anh Tuan Luu, Caiming Xiong, Hae Won Park, Bryan Hooi, Zhiyuan Hu arXiv: 2606.13681

Abstract

Large language model (LLM) agents have achieved strong performance on a wide range of benchmarks, yet most evaluations assume static environments. In contrast, real-world deployment is inherently dynamic, requiring agents to continually align their knowledge, skills, and behavior with changing environments and updated task conditions.

To address this gap, the authors introduce EvoArena, a benchmark suite that models environment changes as sequences of progressive updates across terminal, software, and social domains. They further propose EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental evolution through changes in their memory.

Key Findings

Current agents struggle on EvoArena, achieving an average accuracy of only 39.6% across the evolving terminal, software, and social preference domains.
EvoMem consistently improves performance, with an average gain of 1.5% on EvoArena.
On standard benchmarks, EvoMem improves GAIA by 6.1% and LoCoMo by 4.8%.
EvoMem improves chained accuracy on EvoArena by 3.7%, i.e., when success requires completing consecutive related evolution subtasks.
Mechanistic analysis shows that EvoMem improves document capture in memory, indicating better preservation of complete evolving environment states.

Conclusion

The results highlight the importance of modeling evolution in both evaluation and memory design for reliable real-world agent deployment.