Skill Self-Play (Skill-SP): A Co-Evolutionary Framework Pushing the Frontier of LLM Capability

Forum topic · 小凯 · 2026-07-28

Summary

Skill Self-Play (Skill-SP) is a co-evolutionary training framework for large language models that resolves the tension between task diversity and verification reliability in self-evolving LLM training. Existing environment-bound methods give precise feedback but narrow domain coverage, while open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute training. Skill-SP treats agent skills as a middle ground: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills preserves open-ended task variety. The framework comprises a proposer that generates challenging tasks from dynamically sampled skills, a solver that explores candidate solutions to push capability limits, and a dynamic skill controller that collects execution feedback to update and expand the skill library, all orchestrated in a reinforcement learning self-play loop. Evaluations on tool-use and reasoning benchmarks show Skill-SP consistently raises the performance ceiling of competent backbone models and catalyzes significant recovery of initially misaligned models. Paper: arXiv 2607.22529.

Paper Overview

Field: NLP Authors: Siyuan Huang, Pengyu Cheng, Haotian Liu, Tao Chen, Yihao Liu, Jingwei Ni, Shijie Zhou, Ziyi Yang, Gangwei Jiang, Mengyu Zhou, Yu Cheng, Xiaoxi Jiang, Guanjun Jiang Published: 2026-07-24 arXiv: 2607.22529

Abstract (Translated)

LLM training is shifting from manual design and annotation to interaction-driven self-evolution. However, existing self-evolutionary methods face a fundamental dilemma between task diversity and verification reliability: environment-bound methods obtain precise feedback but confine learning to narrow domains, while open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop.

The authors identify agent skills as a powerful middle ground to reconcile this tension: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills maintains open-ended task variety.

The Skill-SP Framework

Skill Self-Play (Skill-SP) is a co-evolutionary framework comprising three components, orchestrated by a reinforcement learning loop:

Proposer: generates challenging tasks based on dynamically sampled skills
Solver: explores candidate solutions to push the boundary of its capabilities
Skill Controller: collects execution feedback to update and expand the skill library

These components co-evolve in a continuous self-play loop, effectively bridging the gap between structured verification and open-ended exploration.

Results

Empirical evaluation on tool-use and reasoning benchmarks shows that Skill-SP acts as a robust evolution engine that:

Consistently raises the performance ceiling of competent backbone models
Catalyzes significant reversal (recovery) of initially misaligned models