[论文] EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic E...

论文概要

研究领域: NLP
作者: Jundong Xu, Qingchuan Li, Jiaying Wu
发布时间: 2025-06-13
arXiv: 2506.10671

中文摘要

大语言模型(LLM)智能体在各类基准测试中表现强劲，但大多数评估都假设环境是静态的。然而，真实世界的部署本质上是动态的，要求智能体持续调整其知识、技能和行为以适应环境变化和任务条件更新。为填补这一空白，我们推出EvoArena——一个基准测试套件，将环境变化建模为终端、软件和社会领域的渐进式更新序列。我们进一步提出EvoMem，一种基于补丁的记忆范式，将记忆进化记录为结构化更新历史，使智能体能够通过记忆变化推理环境演化。实验表明，当前智能体在EvoArena上表现挣扎，在演化的终端、软件和社会偏好领域平均准确率仅为39.6%。EvoMem持续提升性能，在EvoArena上平均增益1.5%，在GAIA和LoCoMo标准基准上分别提升6.1%和4.8%。在链式任务层面，EvoMem在EvoArena上进一步提升准确率3.7%——成功需要完成一系列相关的演化子任务。机理分析显示EvoMem改善了记忆中的证据捕获，表明更好地保留了完整的演化环境状态。我们的结果强调了在评估和记忆中对演化进行建模对于可靠智能体部署的重要性。

原文摘要

Large language model (LLM) agents have achieved strong performance on a wide range of benchmarks, yet most evaluations assume static environments. In contrast, real-world deployment is inherently dynamic, requiring agents to continually align their knowledge, skills, and behavior with changing environments and updated task conditions. To address this gap, we introduce EvoArena, a benchmark suite that models environment changes as sequences of progressive updates across terminal, software, and social domains. We further propose EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental evolution through changes in their memory. Experiments show that current agents struggle on EvoArena, achieving an averag...

自动采集于 2026-06-14

#论文 #arXiv #NLP #小凯