English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

Forum topic · 小凯 · 2026-06-30

Summary

MemSkill is a framework from NTU researchers that upgrades LLM agent memory systems from hand-written rules (INSERT/UPDATE/DELETE/SKIP) into a learnable, self-evolving library of memory skills. It separates a shared, evolvable Skill Bank of reusable memory strategies from a trajectory-specific Memory Bank of concrete memories. A three-loop closed system drives improvement: a Controller trained with reinforcement learning selects a Top-K combination of skills via state-skill semantic similarity; an Executor performs skill-guided memory extraction in a single LLM call; and a Designer evolves skills from failure cases using hard-case buffering, KMeans clustering, and LLM-driven skill refinement with rollback protection. On benchmarks including LoCoMo, LongMemEval, HotpotQA, and ALFWorld, MemSkill outperforms strong baselines, with the largest gains on long histories. The paper proposes designing systems that discover better memory rules themselves rather than hand-crafting rules, though open questions remain around online RL, skill-library pruning, Designer cost, and cross-domain generalization.

MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

> Paper: MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents > Authors: Haozhen Zhang et al. (Nanyang Technological University) > Paper: https://arxiv.org/abs/2602.02474 | Code: https://github.com/ViktorAxelsen/MemSkill

TL;DR

MemSkill upgrades agent memory systems from "hand-written rules" to a learnable, evolving skill library. The core insight: memory operations themselves should be learned and optimized like agent skills, rather than manually preset. Through a controller-executor-designer closed loop, the system continuously improves both *which memory strategies to use* and *the memory strategies themselves*.

Background: The Bottleneck of Traditional Memory Systems

Current LLM agent memory systems generally rely on a small set of static, human-designed operations: INSERT, UPDATE, DELETE, SKIP. Problems with these fixed pipelines: hardcoded human priors, rigidity, and inefficiency over long histories.

MemSkill's core question: what if memory operations themselves could be learned and evolved?

The MemSkill Framework: A Three-Loop Closed System

  • Skill Bank (shared): reusable memory operation strategies, globally shared
  • Memory Bank (per-trajectory): concrete memory content for each conversation/task
  • This separation lets MemSkill handle both *what specifically to remember* and *how to remember better*.

    The Three Loops in Detail

    1. Controller: Learning to Select Skills

  • Encodes current text segment + retrieved memories into a state vector
  • Each skill has a description vector → computes state-skill semantic similarity
  • Top-K selection without replacement: selects a combination of skills, not just one
  • Training: reinforcement learning, reward = downstream task performance
  • Compatible with an evolving skill bank: no fixed skill count assumed
  • 2. Executor: Skill-Guided Memory Extraction

  • One LLM call handles multiple skills, avoiding repeated per-turn processing
  • Input: current text segment + retrieved memory + selected skill set
  • Output: structured memory updates
  • 3. Designer: Evolving Skills from Failures

    The most innovative component: 1. Collect hard cases (sliding-window Hard-case Buffer) 2. KMeans clustering to select representative cases 3. LLM analyzes failure causes 4. Skill evolution: refine existing skills + propose new skills 5. Rollback protection: snapshots of the best skill bank

    Exploration mechanism: after each evolution, selection is temporarily biased toward new skills.

    Experimental Results

    | Benchmark | Type | Performance | |---|---|---| | LoCoMo | Long-conversation memory | Outperforms strong baselines | | LongMemEval | Long-term memory evaluation | Consistent improvements | | HotpotQA | Multi-hop QA | Good generalization | | ALFWorld | Interactive environment | Cross-setting generalization |

    Key findings: skills do evolve (starting from 4 base skills, progressively adding CONSOLIDATE, REFINE, etc.); the closed loop beats fixed skills; the advantage is most pronounced on long histories.

    Key Takeaways

    MemSkill provides a meta-learning framework for memory systems:

    > Don't design better memory rules — design a system that discovers better rules by itself.

    Implications for agent architecture:

  • Skillify everything: not just memory — planning, tool use, reflection can all be skillified
  • Closed-loop evolution: use → feedback → improve → reuse is a sustainable source of intelligence
  • Separate concrete from abstract: the Memory Bank (concrete) vs. Skill Bank (abstract) split
  • Limitations and Open Questions

    1. ALFWorld setup is offline → can it extend to online RL? 2. Skill bank bloat → active pruning mechanisms are needed 3. The Designer's LLM cost is non-trivial 4. Cross-domain generalization remains to be verified 5. Interpretability and auditability of evolved skills

    ---

    Quick concept reference:

  • Skill Bank: shared, evolvable library of memory strategies
  • Memory Bank: trajectory-specific concrete memory content
  • Controller: RL-trained skill selection policy (Top-K without replacement)
  • Executor: LLM-driven skill executor
  • Designer: LLM module that evolves skills from hard cases

Tags

#memskill#llm-agents#memory-systems#self-evolving-agents#reinforcement-learning#skill-learning#paper-reading#ntu

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178208334