EvoArena: Git-Style Patch Memory for LLM Agents in Dynamic Environments

EvoArena深度分析：当环境不停变，你的Agent记忆还在用"覆盖保存"？

> Paper: EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments > Authors: Jundong Xu, Qingchuan Li, Jiaying Wu, Yihuai Lan, Shuyue Stella Li, Huichi Zhou, Bowen Jiang, Lei Wang, Jun Wang, Anh Tuan Luu, Caiming Xiong, Hae Won Park, Bryan Hooi, Zhiyuan Hu > Institutions: National University of Singapore, Singapore Management University, University of Washington, UCL, UPenn, NTU, Recursive, MIT > Paper: arXiv:2606.13681 > Code: open source https://github.com/Aiden0526/EvoArena > Date: 2026-06-11

One-line summary

EvoArena exposes a fatal blind spot in existing agents: environments change, but memory stores only the latest state. The authors build three dynamic-evolution benchmarks (terminal workflows, codebases, user preferences) and propose EvoMem—a Git-style patch approach that records memory evolution history instead of overwriting. Experiments show an average chain-level accuracy gain of 3.7%, with especially strong effects under continuous environmental change.

Core problem: why agents "forget" once deployed

Existing LLM agent evaluation assumes a static environment: one task, one attempt, one score. But the real world is not like that:

• APIs your scripts depend on change interfaces
• Codebases you maintain keep gaining new versions, features, and bugs
• User preferences drift over time—today's "like" becomes tomorrow's "dislike"

Agents must answer three questions: 1. What changed? (environment evolution detection) 2. What is still valid? (retention of old knowledge) 3. How to act under the current version? (adaptation)

But current agents merge all memory into a single latest state. The authors call this "State Collapse"—previously valid rules/behaviors get overwritten, and the agent fails completely when environments roll back or multiple versions coexist.

EvoArena: three real evolution scenarios

Scenario 1: Terminal-Bench-Evo

• Evolution: terminal dependencies, interfaces, paths, validation rules change continuously
• Scale: 89 initial tasks → 352 evolved versions, 441 instances total
• Tests: whether the agent can keep executing workflows after interface changes

Scenario 2: SWE-Chain-Evo

• Evolution: codebase milestones, new features, bug fixes
• Scale: 50 evolution chains, 493 chain-step instances, 145 unique milestones
• Tests: software engineering tasks on a continuously evolving codebase

Key design: after step t, reference milestone updates (not the agent's own patches) form the repo state for step t+1, isolating "adapting to an evolving repo" from "early error accumulation."

Scenario 3: PersonaMem-Evo

• Evolution: user preferences, styles, temporal validity
• Scale: 10 persona conversations, 505 questions, 313 preference chains
• Tests: tracking temporal trajectories of user preferences

Four question types: single-pattern transfer, multi-pattern synthesis, conflict resolution, and temporal trajectory extrapolation.

EvoMem: Git-like patch memory—no more overwriting

• Append-only, never overwrite—like a Git commit history
• Each patch records 6 elements: timestamp, pre-update content, post-update content, update rationale, semantic summary, supporting evidence
• Reasoning defaults to the latest memory—like a normal agent
• When old-version knowledge is needed—retrieve relevant patches and restore the valid state

Trigger: only "non-additive updates"

Patches are created only when memory is modified, overwritten, or reinterpreted. Purely additive observations do not trigger patch records.

Cross-agent instantiation

| Agent type | Base memory | What patches record | |-----------|---------|-------------| | Terminus2 (terminal agent) | distilled task-solving knowledge | strategy changes from terminal environment shifts | | OpenHands (SE agent) | distilled SE context | superseded implementation strategies, revision logic | | A-Mem (conversational memory agent) | semantic notes and links | note and relation updates | | Memento-S (skill-memory agent) | global reusable skill files | task-specific prompt updates, trigger failures |

Results: how fragile are agents in dynamic environments?

Baselines: poor chain-level accuracy

| Benchmark | Step-level acc. | Chain-level acc. | |------|-----------|-----------| | Terminal-Bench-Evo | 43.6% | 21.5% | | SWE-Chain-Evo | 27.9% | 10.0% | | PersonaMem-Evo | 47.3% | 40.0% |

Chain-level means "correct from start to finish." Baselines can barely stay reliable over long horizons—once one step goes wrong, everything after fails.

EvoMem gains

| Benchmark | Step-level gain | Chain-level gain | Key finding | |------|---------|---------|---------| | Terminal-Bench-Evo | +2.4% | +6.1% | chain-level gain 2.5x step-level | | SWE-Chain-Evo | +0.4% | +2.1% | hardest setting, still improves | | PersonaMem-Evo | +1.7% | +3.2% | temporal trajectory benefits most | | Average | +1.5% | +3.7% | chain-level > step-level |

Best case: Terminal-Bench-Evo + GPT-5.5 goes from 31.8% baseline chain-level to 45.5% with EvoMem—+13.7 points.

Gains on standard benchmarks

| Benchmark | Agent | Baseline | +EvoMem | Gain | |------|-------|------|-------|------| | GAIA | Memento-S | 65.8% | 72.3% | +6.5% | | LoCoMo | A-Mem | 39.7% | 43.0% | +3.3% |

Ablations: what actually works?

Finding 1: patches must be adopted to help. On Terminal-Bench-Evo, when patches are retrieved but not adopted, gains are +2.6%; when retrieved and adopted, +8.3% (88.9% vs 80.6% baseline). The key is operationalizing retrieved transition information, not just adding context.

Finding 2: lower Pass-to-Pass failure on SWE chains (breaking historical behavior when adding new features): Qwen3.6-27B 9.01% → 6.73%; Kimi-K2.6 7.14% → 3.33%.

Finding 3: PersonaMem question types diverge. Multi-pattern synthesis (+5.2%) and temporal trajectory (+5.2%) benefit most; conflict resolution (-0.9%) and single-pattern transfer (-1.8%) slightly decline.

Efficiency-accuracy trade-off: more tokens ≠ better

• PersonaMem-Evo: GPT-5.5 hits 40.3% with 69.2M tokens (highest); Kimi-K2.6 gets 51.5% with only 24.5M; Gemma-4-31B is best at 52.6% with 27.1M
• Terminal-Bench-Evo: GPT-5.5 reaches 62.8% but burns 505.0M tokens; Gemini-3.1 Pro / GLM-5.1 get 53.8%/51.8% with ~80M each

Conclusion: a bigger token budget does not equal better performance. EvoMem lets small/mid-size models use memory history efficiently.

Why this matters

1. State collapse is an implicit bug in systems like Mem0, A-Mem, and LangGraph that assume "latest = best." Merging into a single state loses all context about why things changed and what came before. 2. Chain-level accuracy is the real metric—baseline chain-level accuracy of 10%-40% reveals true deployment fragility. 3. Lightweight and plug-and-play: EvoMem adds a patch layer on top of existing memory systems—no model changes, no fine-tuning, no retraining. 4. Validated across domains: terminal workflows, codebases, and user preferences—suggesting state collapse is a general problem with patch history as a general solution.

Limitations

| Limitation | Notes | |------|------| | Limited evolution scope | no multimodal perception or physical environment changes | | Patch creation overhead | non-additive update detection may bottleneck in high-frequency scenarios | | Retrieval precision | wrong patch retrieval can introduce noise | | Storage growth | long-running agents may need compression/archival strategies | | RAG integration untested | synergy with external vector DBs not evaluated |

One-line takeaway

Agents are not robust because they only remember "now" and forget "what was" and "why it changed." Replace overwrite-saves with Git-style patch history so agents know what changed, what still holds, and how to act under the current version.

> "Memory is not a snapshot—it is an evolution history."

Reference: Jundong Xu et al. "EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments." arXiv:2606.13681, 2026.