CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification

Forum topic · 小凯 · 2026-05-30

Summary

This article summarizes the 2026 arXiv paper CoEvoSkills, which challenges the assumption that human-authored Agent Skills are optimal for LLM agents. The authors, from UIC, MBZUAI, McGill, Columbia, Zhejiang, and UBC, show that manually written Skills can cause human-AI cognitive misalignment, even producing negative returns in domains like Natural Science on the SkillsBench benchmark (87 tasks, 11 fields). The proposed solution is a three-party co-evolutionary framework with information isolation. A Skill Generator produces and refines skill packages using accumulated feedback, a Surrogate Verifier independently generates test assertions and structured diagnostics without seeing the generator's code, and a Ground Truth Oracle issues only opaque binary pass/fail signals to prevent test overfitting. Alternating optimization drives both components to improve, converging within five rounds. Results show 71.1% pass rate on SkillsBench versus 30.6% baseline, a 40.5 percentage-point gain, and 35-44 point improvements when transferred across six LLMs from different vendors. The core insight: AI-evolved skills capture LLM-native reasoning patterns rather than human-intuitive workflows.

Key Points

The Problem with Human-Written Agent Skills

Anthropic's Agent Skills concept packages workflows, executable scripts, and domain references into reusable multi-file templates for LLM agents. However, the CoEvoSkills paper (arXiv, April 2026; authors from UIC, MBZUAI, McGill, Columbia, Zhejiang, UBC) demonstrates that expert-authored skills can actually hurt AI performance due to human-AI cognitive misalignment:

On SkillsBench (87 tasks, 11 professional domains), manual skills delivered inconsistent results
Some domains (e.g., software engineering) saw clear gains
Natural Science showed negative returns — adding skills made performance worse than the no-skill baseline
Root cause: human workflows are optimized for human abstraction, not LLM tool-use patterns

The Three-Party Co-Evolutionary Framework

CoEvoSkills replaces human-in-the-loop authoring with an isolated, adversarial loop:

1. Skill Generator (Student)

Produces candidate skill packages from task instructions
Executes the skill, reads verifier feedback, and iterates
Maintains a persistent dialogue context that accumulates all historical feedback so each new version learns from prior failures

2. Surrogate Verifier (Mentor)

Runs in a completely independent LLM session
Cannot see the generator's code, reasoning, or skill content
Sees only task instructions and output files, then generates test assertions and scores results
Produces structured failure diagnostics (failed assertions, root cause, actionable suggestions) when tests fail
Information isolation prevents confirmation bias — the verifier provides independent, external feedback rather than echoing the generator's own assumptions

3. Ground Truth Oracle (Cold Examiner)

Re-executes the skill package in a fresh environment
Returns only a binary pass/fail signal — no test details, no failure traces, no scoring rubric
Prevents the generator from overfitting to test specifics

The Alternating Optimization Loop

1. Generator produces a skill → executes it → submits output to Verifier 2. Verifier tests output → if fails, returns diagnostic feedback → Generator revises (with fixed test suite) 3. Verifier tests output → if passes, hands off to Oracle 4. Oracle fails → returns only "fail" → Verifier must independently upgrade its test suite (stricter, more diverse assertions) 5. Loop repeats

This creates mutual pressure: the Generator sharpens under Verifier testing while the Verifier hardens its tests under Oracle scrutiny. The system converges within 5 rounds.

Core Results

| Condition | SkillsBench Pass Rate | |---|---| | No-skill baseline | 30.6% | | Background knowledge only (no evolution) | 48.6% | | Verifier removed (Oracle-only opaque signal) | 41.1% | | CoEvoSkills (full framework) | 71.1% | | Human-authored skills | Mixed (negative in some domains) |

Key findings:

+40.5pp over the no-skill baseline
Surpasses human skills within 5 evolution rounds
Removing the Verifier costs 30pp — diagnostic feedback is the fuel for evolution; without it, the generator searches in the dark
Cross-model transfer: the same evolved skill package yields +35 to +44pp lifts when ported to LLMs from six different vendors

Bottom Line

The core insight is not "let AI write its own skills" but "let AI write skills in an AI-native way." Human-authored skills fail not because humans lack intelligence, but because human cognition and LLM cognition are structurally mismatched. AI-evolved skills capture the reasoning patterns and tool-use strategies that LLMs actually need — not the steps humans believe they should take.