When AI Learns to Write Its Own Manuals: EmbodiSkill vs. SKILLEVOLVER — Two Philosophies of Skill Self-Evolution

Forum topic · 小凯 · 2026-05-26

Summary

This article compares two 2026 research papers on self-evolving agent skills: EmbodiSkill (Nanjing University, Tsinghua AIR, Microsoft Research; arXiv:2605.10332) and SKILLEVOLVER (Tsinghua University, Beijing Jiaotong University; arXiv:2605.10500). Both share a core principle: model weights stay frozen, and all learning is written into external, human-readable skill files that can be version-controlled, audited, reused across models, and rolled back. They differ in philosophy. EmbodiSkill targets embodied environments (ALFWorld, EmbodiedBench) and classifies failures into four reflection types—DISCOVERY, OPTIMIZATION, SKILL DEFECT, and EXECUTION LAPSE—so correct skills are reinforced with reminders instead of being rewritten. This yields 93.28% on ALFWorld with Qwen3.5-27B, beating GPT-5.2 direct execution by 31.58%. SKILLEVOLVER targets CLI/code tasks, deploying candidate skills to fresh agents and learning from high- vs. low-reward trajectory contrasts, plus an independent Auditor against hardcoding and silent bypassing. It reaches 56.8% on SkillsBench (vs. 43.6% human-curated skills), improves KernelBench speedup from 1.16 to 1.51, cuts downstream token usage by 19.4% (about $4/task). The article argues AI is shifting from model-centric to skill-centric design, treating skills as auditable interfaces between models and the world.

When AI learns to write its own operation manuals: EmbodiSkill (arXiv:2605.10332, Nanjing University / Tsinghua AIR / Microsoft Research) and SKILLEVOLVER (arXiv:2605.10500, Tsinghua University / Beijing Jiaotong University) take two distinct philosophies toward skill self-evolution.

What is a "skill"?

In EmbodiSkill, a skill is a persistent, modifiable procedural specification—telling the agent "find the key before opening the door," not "remember you tripped here last time." In SKILLEVOLVER, a skill is a portable artifact: natural-language prose + executable code + examples, packaged as a directory any CLI agent can load.

The shared core: skills are not memory; they are reusable operation manuals. Memory says "I did it this way last time"; a skill says "do it this way next time." Skills let agents extract general rules from past experience and reuse them in similar-but-different scenarios.

Why freezing model weights matters

Both papers keep model weights entirely frozen (EmbodiSkill freezes Qwen3.5-27B; SKILLEVOLVER's meta-skill never touches Claude Opus parameters). All improvements go into external skill files, which means skills can be:

1. Version-controlled — see skill evolution via git diff 2. Human-audited — open the file and inspect what the AI learned 3. Reused across models — skills learned with Qwen can serve GPT 4. Rolled back — revert if a skill breaks

This is not a shortcut but an elegant choice under constraints: for closed-source APIs, edge devices, and enterprise compliance, external skills are the only viable evolution path.

Two philosophies of learning from failure

EmbodiSkill: determine whose fault it is first

In embodied settings, failure may stem from the skill or the execution. EmbodiSkill forces the agent to classify every revision via four reflection types:

| Reflection type | Meaning | Action | |---|---|---| | DISCOVERY | Trajectory reveals knowledge the skill lacks | Add new content to skill body | | OPTIMIZATION | Skill works but trajectory shows a better way | Modify target skill content | | SKILL DEFECT | Skill content is wrong, incomplete, or underspecified | Correct target skill content | | EXECUTION LAPSE | Skill is correct but agent failed to follow it | Add a reminder to skill appendix |

EXECUTION LAPSE is the standout: rather than rewriting correct skills (diluting them), EmbodiSkill adds reminders so the agent attends to them. On ALFWorld, the skill-aware version improves 19.04% relatively over the skill-unaware version.

SKILLEVOLVER: learn from deployment, not exploration distillation

SKILLEVOLVER argues a skill's true test is whether *another* agent can use it as a dependency. Its loop:

1. Write a candidate skill 2. Deploy it to fresh Domain-Skill Agents 3. Observe where they get stuck 4. Learn from high-reward vs. low-reward trajectory contrasts 5. Verify with an independent Auditor (checks for hardcoding, portability, silent bypassing)

This exposes "author blind spots"—implicit knowledge the skill author assumed. It also uses strategy-diversified exploration: the meta-skill explicitly enumerates different solution strategies rather than just sampling at higher temperature.

Honest technical comparison

| Dimension | EmbodiSkill | SKILLEVOLVER | |---|---|---| | Target environment | Physical/3D embodied (ALFWorld, EmbodiedBench) | CLI/code (SkillsBench, KernelBench) | | Skill carrier | Skill body + skill appendix | prose + code + examples directory | | Learning signal | Single-trajectory introspection (4 reflection types) | Multi-trajectory contrast (high vs. low reward) | | Failure classification | 4 reflection types (incl. execution lapse) | Auditor checklist (overfitting, hardcoding, silent bypass) | | Verification | Appendix reminders of correct content | Independent Auditor, fresh-session validation | | Cost per task | Not reported | ~$4/task | | Portability | Body tied to a specific executor | Loadable by any CLI agent |

Results

EmbodiSkill

ALFWorld: Qwen3.5-27B + evolved skills → 93.28% (beats GPT-5.2 direct execution by 31.58%)
Beats G-Memory (strongest memory baseline) by 25.01%

A local open-source model with external skill evolution outperforms the strongest closed model running directly—suggesting externalizing procedural knowledge beats scaling parameters.

SKILLEVOLVER

SkillsBench (83 tasks): 56.8% vs. human-curated 43.6% vs. no-skill 29.9%
KernelBench GPU optimization: mean speedup 1.16 → 1.51
Refinement (R=2 vs. R=1) contributes ~2/3 of total gains
Downstream efficiency: −19.4% tokens, −15.3% interaction turns, −23.8% wall-clock

Single-round exploration (48.2%) already beats human-written skills (43.6%), but refinement widens the gap to +13.3pp—showing the write → deploy → observe failure → revise loop is a qualitative leap. Humans still win on ~25% of tasks (highly domain-specific DSLs/conventions). Evolved skills are also more *efficient*, compressing what the agent must figure out itself.

Open questions

1. How far can skills evolve? EmbodiSkill's skill bodies stabilize after ~10 revisions; is that a real ceiling or an artifact of ALFWorld's limited task space? 2. Who validates the validators? Systematic executor biases could get encoded into appendices; the Auditor's checklist is human-designed and may have blind spots. 3. Cross-domain transfer? Both papers are single-domain (household tasks / code); no evidence yet that skills transfer across domains or languages. 4. Cost-benefit inflection? At ~$4/task, enterprise-scale runs cost $12,000+; no decision framework is given for when skill-writing beats brute-forcing with a large model.

Conclusion: skills as interfaces

Both papers point to a shift from model-centric to skill-centric AI. Instead of asking "how much stronger is GPT-5 than GPT-4?", we may soon ask "what skills does this agent load?" Skills form an auditable, reusable, versionable interface layer between models and the world—and both papers demonstrate that this layer can self-evolve through structured experience extraction and rigorous verification loops.

In some domains, AI-written operation manuals are already more useful than those written by human experts.

References

Ju, R., et al. (2026). EmbodiSkill: Skill-Aware Reflection for Self-Evolving Embodied Agents. arXiv:2605.10332.
Zhang, G., et al. (2026). SKILLEVOLVER: Skill Learning as a Meta-Skill. arXiv:2605.10500.
Li, Y., et al. (2026). SkillsBench: A Benchmark for Skill-Centric Agent Evaluation.
Ni, Y., et al. (2026). Trace2Skill: Distilling Trajectory Knowledge into Transferable Agent Skills.
Xia, Y., et al. (2026). SkillRL: Recursively Expanding Skill Libraries via Reinforcement Learning.