MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
> Paper: MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents > Authors: Haozhen Zhang et al. (Nanyang Technological University) > Paper: https://arxiv.org/abs/2602.02474 | Code: https://github.com/ViktorAxelsen/MemSkill
TL;DR
MemSkill upgrades agent memory systems from "hand-written rules" to a learnable, evolving skill library. The core insight: memory operations themselves should be learned and optimized like agent skills, rather than manually preset. Through a controller-executor-designer closed loop, the system continuously improves both *which memory strategies to use* and *the memory strategies themselves*.
Background: The Bottleneck of Traditional Memory Systems
Current LLM agent memory systems generally rely on a small set of static, human-designed operations: INSERT, UPDATE, DELETE, SKIP. Problems with these fixed pipelines: hardcoded human priors, rigidity, and inefficiency over long histories.
MemSkill's core question: what if memory operations themselves could be learned and evolved?
The MemSkill Framework: A Three-Loop Closed System
- Skill Bank (shared): reusable memory operation strategies, globally shared
- Memory Bank (per-trajectory): concrete memory content for each conversation/task
- Encodes current text segment + retrieved memories into a state vector
- Each skill has a description vector → computes state-skill semantic similarity
- Top-K selection without replacement: selects a combination of skills, not just one
- Training: reinforcement learning, reward = downstream task performance
- Compatible with an evolving skill bank: no fixed skill count assumed
- One LLM call handles multiple skills, avoiding repeated per-turn processing
- Input: current text segment + retrieved memory + selected skill set
- Output: structured memory updates
- Skillify everything: not just memory — planning, tool use, reflection can all be skillified
- Closed-loop evolution: use → feedback → improve → reuse is a sustainable source of intelligence
- Separate concrete from abstract: the Memory Bank (concrete) vs. Skill Bank (abstract) split
- Skill Bank: shared, evolvable library of memory strategies
- Memory Bank: trajectory-specific concrete memory content
- Controller: RL-trained skill selection policy (Top-K without replacement)
- Executor: LLM-driven skill executor
- Designer: LLM module that evolves skills from hard cases
This separation lets MemSkill handle both *what specifically to remember* and *how to remember better*.
The Three Loops in Detail
1. Controller: Learning to Select Skills
2. Executor: Skill-Guided Memory Extraction
3. Designer: Evolving Skills from Failures
The most innovative component: 1. Collect hard cases (sliding-window Hard-case Buffer) 2. KMeans clustering to select representative cases 3. LLM analyzes failure causes 4. Skill evolution: refine existing skills + propose new skills 5. Rollback protection: snapshots of the best skill bank
Exploration mechanism: after each evolution, selection is temporarily biased toward new skills.
Experimental Results
| Benchmark | Type | Performance | |---|---|---| | LoCoMo | Long-conversation memory | Outperforms strong baselines | | LongMemEval | Long-term memory evaluation | Consistent improvements | | HotpotQA | Multi-hop QA | Good generalization | | ALFWorld | Interactive environment | Cross-setting generalization |
Key findings: skills do evolve (starting from 4 base skills, progressively adding CONSOLIDATE, REFINE, etc.); the closed loop beats fixed skills; the advantage is most pronounced on long histories.
Key Takeaways
MemSkill provides a meta-learning framework for memory systems:
> Don't design better memory rules — design a system that discovers better rules by itself.
Implications for agent architecture:
Limitations and Open Questions
1. ALFWorld setup is offline → can it extend to online RL? 2. Skill bank bloat → active pruning mechanisms are needed 3. The Designer's LLM cost is non-trivial 4. Cross-domain generalization remains to be verified 5. Interpretability and auditability of evolved skills
---
Quick concept reference: