Skill Self-Play: When AI Learns to Play Against Itself to Co-Evolve Skills

*English translation of a forum post interpreting the paper "Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills" (arXiv:2607.22529).*

Opening: A Solo Game with No Audience

The post opens with an image: a Go player late at night, playing black and white against themselves, learning from every move. This is the paper's central metaphor — large language models (LLMs) engaging in "self-play," not over a board game, but over skills, through a framework called Skill Self-Play (Skill-SP).

Chapter 1: From Supervised Training to Self-Evolution

Traditional LLM training relies heavily on humans:

• Supervised learning: human annotators provide questions and correct answers.
• Reinforcement learning: humans define reward rules and evaluation criteria.

Both keep AI in the role of a student whose curriculum is fully arranged by others. Self-evolution removes the human from the loop with a cycle of: generate tasks → solve → evaluate → improve → repeat.

But this loop faces a fundamental dilemma the paper identifies:

> Environment-constrained approaches obtain precise feedback but confine learning to narrow domains; open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop.

The author illustrates with a cooking example: a strict cooking school (closed) makes correctness easy to check but limits the menu; free-form culinary creativity (open) has no ground truth for "is this dish good?"

Chapter 2: Skills as the Bridge Between Closed and Open

Skills are proposed as the sweet spot between the two extremes, compared to a Swiss Army knife:

• Each skill is verifiable — it has clear success criteria in specific scenarios.
• Skills can be dynamically combined via routing.
• The skill library can grow when new situations appear.

A second analogy: a restaurant kitchen. Each station (prep cook, wok chef, pastry chef) has a verifiable standard operating procedure = a skill; the head chef assigning dishes per order = dynamic routing; the whole kitchen running = the Skill-SP framework.

Chapter 3: The Three-Dancer Loop

Skill-SP has three co-evolving components:

1. Proposer — generates tasks targeted at the current skill library, deliberately increasing difficulty to push the solver out of its comfort zone. Like a martial arts coach designing sparring scenarios that force the student to practice weak techniques. 2. Solver — attempts the tasks, exploring candidate solutions and pushing its capability boundary, sometimes inventing new approaches. 3. Skill Controller — the conductor: observes the solver's performance, decides which skills to retain/strengthen, which to improve, and where new skills are needed.

All three are coupled via reinforcement learning, where rewards come from mutual interaction rather than human judgment:

• A well-designed challenging-yet-solvable task rewards the Proposer.
• Solving a hard task rewards the Solver.
• Correctly identifying blind spots rewards the Skill Controller.

This creates a self-driven cycle: propose → solve → evaluate → update the skill library → repeat with a stronger library.

Chapter 4: Experimental Results

The paper evaluates Skill-SP on tool-use benchmarks (calculator, search, code execution, etc.) and reasoning benchmarks, across multiple backbone models — both already-strong models and weak ones.

Key findings:

• Strong models keep improving: Skill-SP acts as a "robust evolution engine" that consistently pushes the performance ceiling.
• Weak models turn around dramatically: initially misaligned models show "striking turnarounds" without heavy human annotation or adjustment.

The post infers a three-stage growth trajectory for the skill library: basic single-skill accumulation → compositional skills and routing → complex strategies and "meta-skills" (knowing when to use which skill, and creating new combinations).

Chapter 5: What It Means

• From using tools to creating tools: the Skill Controller exhibits a form of metacognition — recognizing its own blind spots and driving the system to learn new skills to fill them.
• Autonomy and its limits: the whole loop — proposing, solving, evaluating, improving — involves no humans. Quoting Feynman: "What I cannot create, I do not understand." Skill-SP closes the loop between creating (new skills) and verifying (their effectiveness), though it still operates within domains with clear evaluation criteria.
• Comparison with AlphaGo: AlphaGo self-plays under fixed rules; Skill-SP self-plays under constantly expanding rules, growing its own skill library — a "meta-level" evolution that may be a step toward more general AI.

Closing

The post concludes that the future of AI may depend less on how much data humans feed it, and more on whether we can design mechanisms for it to evolve itself — like the midnight Go player, whose real growth happened in an unseen, unapplauded solo game.

Reference

Siyuan Huang, Pengyu Cheng, Haotian Liu, Tao Chen, Yihao Liu, Jingwei Ni, Shijie Zhou, Ziyi Yang, Gangwei Jiang, Mengyu Zhou, Yu Cheng, Xiaoxi Jiang, Guanjun Jiang. "Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills." arXiv:2607.22529, 2026.