Mem²Evolve: Co-Evolutionary Capability Expansion and Experience Distillation for Self-Evolving LLM Agents — In-Depth Critical Review

Forum topic · ✨步子哥 · 2026-07-11

Summary

Mem²Evolve (Cheng et al., ACL 2026, arXiv:2604.10923) proposes a co-evolutionary self-evolution paradigm for LLM agents that couples capability expansion (dynamically creating MCP-compliant tools and expert agents stored in an Asset Memory) with experience distillation (strategic and implementation lessons distilled into an Experience Memory). A forward inference loop reuses or creates assets on demand, while a backward loop uses LLM-as-a-Judge to evaluate trajectories and evolve both memories. Under a single GPT-5-chat backbone with web retrieval and sandboxed execution, the system achieves 70.24% average Pass@1 across 11 benchmarks, outperforming ReAct (+21.97), experience-centric AFLOW (+11.80), and capability-centric Alita (+6.46), with GAIA total of 76.31%. Experience-guided tool creation raises first-pass rates from 53.1% to 72.4%. This critical review verifies the evidence but flags limits: ablations never isolate web/sandbox contributions, no significance tests or cost accounting exist, gains concentrate on retrieval-heavy tasks, and the abstract's "+18.53%" baseline is missing from the main table. A mandatory revision checklist is provided.

Mem²Evolve: Co-Evolutionary Self-Evolving Agents — Deep Research Report

> Research subject: Cheng et al. (2026). *Mem²Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation*. ACL 2026, arXiv:2604.10923v1 > Method: six-phase deep-research pipeline (Scoping → Investigation → Analysis → Composition → Review → Revision); all figures cross-verified against the arXiv full text via two independent fetches.

Abstract of the Framework

Mem²Evolve argues that existing self-evolving agent frameworks treat two evolution paths in isolation:

Experience-centric evolution (DyLAN, DSPy, AFLOW, AgentSquare): optimizes within a fixed, human-predefined tool/role space — experience alone cannot break the capability boundary.
Capability-centric evolution (Alita, AgentVerse, AutoAgents): creates new tools/agents from scratch — without experiential guidance, successes are hard to reproduce and errors repeat.

Mem²Evolve couples the two via a dual-memory architecture:

Asset Memory ℳ_A = ℬ_agt ∪ ℬ_tool: an Agent Bank (role / expertise / behavioral advice / available tools) and a Tool Bank (MCP-compliant tools with name, functional description, implementation code, I/O docs).
Experience Memory ℳ_E: distilled entries e = ⟨h_title, d_desc, 𝒰_case, κ_content⟩, split into *Agent Experience* (strategy) and *Tool Experience* (implementation guidance).

Forward inference (plan → recruit → execute) uses a reuse-first policy: if similarity to Asset Memory exceeds threshold δ, existing assets are recruited; otherwise new tools are created conditioned on retrieved experience plus web knowledge (Serper + Crawl4AI), gated by an "Experience + Web + Self-test" guardrail — synthetic test cases generated via critique must all pass before a tool enters the bank.

Backward evolution applies LLM-as-a-Judge r_t, c_t = Judge(q_t, τ_t, a_t): validated new assets are merged into Asset Memory (with an Improve loop otherwise); reflections over trajectories yield success-generalization strategies and failure-diagnosis anti-patterns for Experience Memory.

Key Results (GPT-5-chat backbone, Pass@1)

| Method | GAIA Tot | ALFWorld | HotpotQA | AIME24 | AIME25 | Avg | |---|---|---|---|---|---|---| | GPT-5-Chat (ReAct) | 18.47 | 86.87 | 41.40 | 66.67 | 60.00 | 48.27 | | AFLOW (Exp-centric) | 19.75 | 93.40 | 60.80 | 66.67 | 63.33 | 58.44 | | Alita (Cap-centric) | 72.73 | 86.13 | 58.80 | 70.00 | 66.67 | 63.78 | | Mem²Evolve | 76.31 | 94.31 | 60.80 | 76.70 | 73.33 | 70.24 |

Average 70.24%: +21.97 over ReAct, +11.80 over AFLOW, +6.46 over Alita. GAIA total 76.31% exceeds the OpenAI Deep Research reference (67.36%) — though this is a cross-system, cross-backbone comparison.
Gains concentrate almost entirely on retrieval/tool-heavy GAIA tasks; HotpotQA is exactly tied with AFLOW and ALFWorld improves by only +0.91 (noise-level).

Ablations (Avg Pass@1)

| Configuration | Avg | Δ | |---|---|---| | Full | 70.24 | – | | w/o Tool Creation | 59.96 | ↓10.28 | | w/o Agent Memory | 65.51 | ↓4.73 | | w/o Tool Memory | 67.11 | ↓3.13 | | w/o Expert Agent Creation | 68.52 | ↓1.72 |

Dynamic tool creation is the most critical component; ablations report only means, not per-task distributions.

Experience-Guided Creation

Experience guidance raises tool-creation first-pass rates from 53.1% to 72.4% (relative +36.3%) and cuts average debug iterations from 1.01 to 0.48 (−52.5%) — the most direct evidence for the "experience guides capability creation" claim.

Transfer

Cross-task initialization with GAIA-derived memory consistently helps on 7 other benchmarks (~25% of the benefit of single-task init), with no negative transfer.
Within-task continuous evolution shows diminishing marginal returns after early gains.

Critical Assessment

Established concerns: 1. All controlled comparisons use a single GPT-5-chat backbone; whether gains generalize to other backbones is untested. 2. Web retrieval and sandbox self-testing are retained in every ablation, so the contribution of "executable tools + live retrieval" vs. memory co-evolution cannot be isolated. The largest ablation drop (tool creation) is exactly the component most dependent on the sandbox loop. 3. Co-evolution coupling is asserted but never measured; the system may effectively be "Alita + an experience layer," with δ-routing as a scheduling heuristic. 4. No error bars, variance, sample sizes, or significance tests anywhere; no cost/token accounting, so "more compute and calls" remains an unfalsified alternative explanation. A judge-filtering effect also cannot be ruled out. 5. 🔴 Critical disclosure inconsistency: the abstract claims "+18.53% over standard LLMs" (implying a baseline ≈51.71) that does not appear in Table 2; the closest non-agent baseline, GPT-5-ReAct at 48.27, implies +21.97. This must be corrected.

Paper-acknowledged limitations: reliance on sandboxed execution of self-generated code, and an LLM-Judge as the single trust source of the backward loop.

Conclusion

Mem²Evolve demonstrably delivers a strong system under GPT-5-chat + web + sandbox conditions, with hierarchical ablations confirming each memory component contributes. However, the evidence supports "the dual-memory system works," not the stronger claim that "the co-evolutionary paradigm is intrinsically superior" — the coupling mechanism's causal contribution is unisolated, gains are task-concentrated, and gains risk being explained by more compute, more retrieval, and selective reporting. Recommended revisions: per-task ablation distributions, no-web/no-sandbox clean ablations, multi-backbone comparisons, cost accounting, multi-run variance with significance tests, long-horizon memory stability curves, and correction of the abstract's baseline disclosure.

References

Cheng, Z., et al. (2026). *Mem²Evolve: Towards self-evolving agents via co-evolutionary capability expansion and experience distillation*. ACL 2026. https://arxiv.org/abs/2604.10923
Qiu, et al. (2025). *Alita*. https://arxiv.org/abs/2505.20286
Zhang, et al. (2025). *AFLOW*. ICLR 2025. https://arxiv.org/abs/2410.10762
Shang, et al. (2024). *AgentSquare*. https://arxiv.org/abs/2410.06153
Khattab, O., et al. (2024). *DSPy*. ICLR 2024. https://arxiv.org/abs/2310.03714
Yan, S., et al. (2026). *Memory-R1*. ACL 2026.
OpenAI. (2025). *Introducing deep research*. https://openai.com/index/introducing-deep-research

*Disclosure: this report was produced with an AI-assisted research pipeline (six-phase, multi-sub-agent); all figures trace to the primary paper and were cross-verified. Speculative critical conclusions are explicitly labeled.*