English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

CoEvoSkills: Self-Evolving Agent Skills via a Three-Player Co-Evolutionary Verification Game

Forum topic · 小凯 · 2026-05-30

Summary

CoEvoSkills is an arXiv paper (April 2026) from researchers at UIC, MBZUAI, McGill, Columbia, Zhejiang, and UBC that addresses a key weakness in Anthropic's Agent Skills concept: manually written skill packages often suffer from human-machine cognitive misalignment, and on SkillsBench (87 tasks, 11 domains) some human-written skills even yield negative gains. CoEvoSkills lets AI autonomously evolve its own skill packages through a three-player game with information isolation: a Skill Generator maintains a persistent context and iteratively improves skills; a Surrogate Verifier, running in a fully separate LLM session, generates test assertions and structured failure diagnostics without seeing the generator's internals, avoiding confirmation bias; and a Ground Truth Oracle re-executes skills in fresh environments, returning only a binary pass/fail signal to prevent overfitting to tests. Alternating optimization drives both generator and verifier to co-evolve, converging within five rounds. Results: 71.1% pass rate on SkillsBench versus a 30.6% no-skill baseline (+40.5pp), with the verifier ablation dropping to 41.1%, and evolved skills transferring across six LLM vendors with +35-44pp gains.

CoEvoSkills (Self-Evolving Agent Skills via Co-Evolutionary Verification) is a paper published on arXiv in April 2026, with authors from six universities including UIC, MBZUAI, McGill, Columbia, Zhejiang University, and UBC. Its core question: after Anthropic introduced the Agent Skills concept, hand-writing skills proved labor-intensive and suffered from human-machine cognitive misalignment — guides written by human experts sometimes work *worse* for AI. Can AI evolve its own skills instead? The paper's answer: a three-player game co-evolutionary framework that autonomously produces skill packages better than human-written ones.

From Tools to Skills: Why Single Functions Are No Longer Enough

Existing LLM agent tools are single function calls — check the weather, do math, read a file. But real-world professional tasks are multi-step, multi-file, cross-tool orchestrations: fixing complex software bugs requires reading code, writing tests, running debuggers, and verifying fixes; scientific analysis requires loading data, cleaning, modeling, visualizing, and writing reports. These tasks cannot be solved by "calling one tool"; they need a structured workflow.

Anthropic's Agent Skills are designed for exactly this: a Skill is not a single function but a multi-file structured package — containing workflow instructions, executable scripts, and domain references. It is like a mini project template: given the package, the agent knows how to complete a class of tasks.

Why Human-Written Skills Can Make AI Worse

The paper tested carefully human-crafted skills on SkillsBench (87 tasks, 11 professional domains). The results are counterintuitive:

  • Some domains (e.g., software engineering) show clear gains
  • Natural Science even shows negative returns — with the skill package, the AI performs worse than without it
  • The authors attribute this to human-machine cognitive misalignment: workflows and abstraction levels designed by human experts are meant for human understanding and execution, but LLM agents reason, handle context, and call tools very differently. Steps that feel "intuitive" to humans may be information overload or wrongly ordered for an AI.

    This raises a fundamental question: if human-written skills are not necessarily good for AI, who should write skills?

    A Three-Player Game: Student, Mentor, and Stern Examiner

    CoEvoSkills has three information-isolated components:

    ① Skill Generator (the student)

    Generates and iterates on skill packages. Starting from a task instruction, it produces a candidate skill package, executes it, and obtains outputs. It then reads the verifier's feedback and improves the skill in the next iteration. Crucially, it maintains a persistent conversation context that accumulates all historical verification feedback — each new version builds on past failures.

    ② Surrogate Verifier (the mentor)

    Runs in a completely independent LLM session, seeing none of the generator's code, reasoning, or skill content. It sees only the task instruction and the generator's output files. From these, it generates test assertions and scores the outputs. On test failure, it produces structured failure diagnostics (which assertions failed, root-cause analysis, actionable fixes) and feeds them back to the generator.

    Information isolation is the soul of this design. If the verifier could see the generator's code, it would inherit the generator's biases — whatever the generator believes is correct, the verifier would also accept. This is confirmation bias, the fatal flaw of self-verification systems. Isolation ensures the verifier provides independent, external, potentially contradictory feedback.

    ③ Ground Truth Oracle (the stern examiner)

    Independently re-executes the skill package in a fresh environment and returns only a binary pass/fail signal. No test contents, failure details, or grading criteria are revealed. This prevents the generator from overfitting to the tests — if it knew what the tests checked, it might optimize for "passing tests" rather than "solving the problem correctly."

    Alternating Optimization: Two Feedback Loops Drive Evolution

    The pipeline alternates:

    1. Generator produces a skill → executes → output goes to the Verifier 2. Verifier tests: if fail → diagnostic feedback → Generator improves the skill (test suite fixed) 3. Verifier tests: if pass → handed to the Oracle 4. If the Oracle fails it → only "fail" is returned (no details) → the Verifier must independently upgrade its test suite (stricter, more diverse tests) 5. Back to step 1, looping

    The elegance: Generator and Verifier pressure each other. The Generator grows stronger under the Verifier's test pressure; the Verifier escalates test difficulty under the Oracle's "leak" signals. The two co-evolve, converging within 5 rounds.

    Core Results: The Numbers

    | Condition | SkillsBench pass rate | |------|-------------------| | No-skill baseline | 30.6% | | Background knowledge only (no evolution) | 48.6% | | Verifier removed (Oracle's opaque signal only) | 41.1% | | CoEvoSkills (full framework) | 71.1% | | Carefully human-written skills | Mixed (negative returns in some domains) |

    Key findings:

  • +40.5pp over the no-skill baseline
  • Surpasses human-written skills within 5 evolution rounds
  • Removing the Verifier costs 30pp: diagnostic feedback is the fuel of evolution; without it, the generator gropes in the dark
  • Cross-model transfer: the same evolved skill package, applied to LLMs from 6 different vendors, delivers +35-44pp gains
---

> Core insight: CoEvoSkills is not just "letting AI write its own skills" — it is letting AI write skills the way AI thinks. Human-written skills fail not because humans aren't smart, but because human cognitive structure doesn't match LLMs. Skills evolved by AI capture the reasoning patterns and tool-use strategies LLMs actually need, not the steps humans think *should* exist.

Tags

#ai-agents#agent-skills#coevolution#llm#automated-verification#skillsbench#self-improving-ai

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177980605