MDSKILL: Multi-View Distillation for Skill Memory Internalization in Self-Evolving LLM Agents

Forum topic · 小凯 · 2026-11-04

Summary

MDSKILL (arXiv:2611.05313) is a multi-view distillation method that helps self-evolving LLM agents internalize skill memories efficiently. Skill memories encode past experiences as textual prompts to improve reinforcement learning, but self-generated memories often contain noisy, redundant, or conflicting knowledge that is hard to distill directly under limited training budgets. MDSKILL organizes skill memories into three complementary functional views — state grounding, action planning, and execution control — selects high-quality skills via dynamic metrics, and builds view-specific and integrated reasoning branches. On-Policy Self-Distillation (OPSD) then consolidates complementary experiential knowledge across these branches into a shared policy. Evaluated on three agent benchmarks against RL, skill-augmented, and self-distillation baselines without stronger external teacher models, MDSKILL improves SOTA performance by up to 7.1% on ALFWorld and 8.1% on WebShop with no additional computational overhead.

Paper Overview

Field: NLP

Authors: Dian Jin, Jinluan Yang, Kairong Han, Baohong Li, Zijing Hu, Yi He et al.

Published: 2026-11-03

arXiv: 2611.05313

Abstract

Skill memories encode past experiences as textual prompts to improve reinforcement learning (RL) for self-evolving agents. Complementing RL, On-Policy Self-Distillation (OPSD) provides dense, fine-grained supervision to internalize knowledge from skill-guided trajectories and accelerate policy learning. However, self-generated memories often contain noisy supervision and functionally entangled, redundant, or conflicting knowledge, making direct distillation difficult. Under limited training budgets, these issues can hinder knowledge internalization and even degrade performance.

Method

To address this problem, the authors propose MDSKILL, a multi-view distillation approach for skill memory internalization:

Organizes skill memories into three complementary functional views: state grounding, action planning, and execution control
Selects high-quality skills using dynamic metrics to construct view-specific and integrated reasoning branches
Applies OPSD to transfer and consolidate complementary experiential knowledge across these branches into a shared policy, enabling efficient skill internalization and continuous agent self-evolution

Results

Evaluated on three agent benchmarks against RL, skill-augmented, and self-distillation baselines that do not rely on stronger external teacher models
Without additional computational overhead, MDSKILL improves upon SOTA performance by up to 7.1% on ALFWorld and 8.1% on WebShop
Extensive ablations and evolution analyses validate its effectiveness in knowledge internalization and policy improvement

Code

https://anonymous.4open.science/r/MDSKILL-CF53