MemSkill: Self-Evolving Memory Skills for LLM Agents

Forum topic · 小凯 · 2026-06-30

Summary

MemSkill is a framework from Nanyang Technological University that replaces hand-crafted memory operations in LLM agents with a learnable, evolving skill library. Built on a three-component loop (Controller, Executor, Designer), it separates a shared Skill Bank of reusable memory strategies from per-trajectory Memory Bank content. The Controller uses reinforcement learning to select top-K skills via semantic similarity, the Executor executes the chosen skills in one LLM call, and the Designer evolves the skill library by clustering hard cases, analyzing failures, and proposing refinements or new skills. Experiments on LoCoMo, LongMemEval, HotpotQA, and ALFWorld show consistent gains over strong baselines, with the largest improvements on long histories. The paper suggests a broader meta-learning paradigm: design systems that discover better rules, rather than hand-designing them.

Overview

MemSkill upgrades the memory system of LLM agents from fixed, hand-written operations (INSERT, UPDATE, DELETE, SKIP) to a library of learnable, evolving memory skills. The core insight is that the memory operations themselves should be learned and optimized like agent skills, not hard-coded as human priors.

Paper: MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
Authors: Haozhen Zhang et al., Nanyang Technological University
Link: https://arxiv.org/abs/2602.02474
Code: https://github.com/ViktorAxelsen/MemSkill

The Problem with Traditional Memory Systems

Current LLM-agent memory systems rely on a small set of static, human-designed operations. Problems include:

Hard-coded human priors
Rigidity across tasks
Inefficiency on long interaction histories

MemSkill's central question: what if memory operations could be learned and evolved?

MemSkill Framework: A Three-Loop Closed Cycle

MemSkill separates two concerns:

Skill Bank — a global, shared, reusable library of memory-operation strategies
Memory Bank — per-trajectory concrete memory content

This separation lets the system handle both *what* to remember and *how to remember better*.

The Three Components

1. Controller — Learning to Select Skills

Encodes the current text chunk plus retrieved memories into a state vector
Each skill is represented by a description vector; state-skill similarity is computed
Top-K sampling without replacement — selects a *combination* of skills, not just one
Trained with reinforcement learning using downstream task performance as the reward
Compatible with an evolving skill bank: no assumption of a fixed skill count

2. Executor — Skill-Guided Memory Extraction

A single LLM call handles multiple skills, avoiding per-turn redundancy
Inputs: current text chunk + retrieved memories + selected skills
Outputs: structured memory updates

3. Designer — Evolving Skills from Failures

The most innovative component. Steps:

1. Collect hard cases via a sliding-window Hard-case Buffer 2. Cluster with KMeans to pick representative failures 3. Use an LLM to analyze failure causes 4. Skill evolution: refine existing skills + propose new skills 5. Rollback protection: snapshot the best-performing skill bank

An exploration bonus biases selection toward newly added skills for a short window after each evolution.

Experimental Results

| Benchmark | Type | Result | |------------|------|--------| | LoCoMo | Long-conversation memory | Beats strong baselines | | LongMemEval | Long-horizon memory eval | Consistent gains | | HotpotQA | Multi-hop QA | Good generalization | | ALFWorld | Interactive environment | Cross-setting generalization |

Key findings:

Skills genuinely evolve: starting from 4 base skills, the system progressively adds CONSOLIDATE, REFINE, and others
The closed loop outperforms fixed skill sets
Advantages are most pronounced on long histories

Core Takeaways

MemSkill provides a meta-learning framework for memory systems:

> Rather than designing better memory rules, design a system that can discover better rules itself.

Implications for agent architecture:

Skill-ify everything — planning, tool use, reflection, etc. can follow the same pattern
Closed-loop evolution — use → feedback → improve → reuse is a sustainable source of intelligence
Separate the concrete from the abstract — Memory Bank (content) vs. Skill Bank (strategy)

Limitations and Open Questions

1. ALFWorld is an offline setting — can the approach scale to online RL? 2. Skill bank bloat — needs active pruning mechanisms 3. Designer LLM cost is non-trivial 4. Cross-domain generalization remains to be verified 5. Interpretability and auditability of evolved skills

Glossary

Skill Bank — shared, evolvable library of memory strategies
Memory Bank — per-trajectory concrete memory content
Controller — RL-trained skill selector (Top-K without replacement)
Executor — LLM-driven skill executor
Designer — LLM module that evolves skills from hard cases