OpenSkill: Open-World Self-Evolution for LLM Agents

> Paper: OpenSkill: Open-World Self-Evolution for LLM Agents > Authors: Zhiling Yan, Dingjie Song, Hanrong Zhang, Wei Liang, Yuxuan Zhang, Yutong Dai, Lifang He, Philip S. Yu, Ran Xu, Xiang Li, Lichao Sun > Link: https://arxiv.org/abs/2606.06741

The Problem: Agents Break in the Open World

LLM agents excel in closed-world settings: clear task boundaries, standard answers, and automated verifiers let them improve through repeated trial and error. But the real world offers none of these — no ground truth, no auto-grader, no human feedback. Existing skill-learning frameworks rely on human-written verifiers, target-task reference answers, or human supervision, all of which are unavailable in open-world environments.

The OpenSkill team poses a sharp question:

> Can an LLM agent learn to solve problems on its own — with no initial skills, no verifiers, no human supervision, and no reference answers?

Their answer: yes, and it outperforms closed-world baselines.

Closed World vs. Open World

| Dimension | Closed World (mainstream methods) | Open World (OpenSkill) | |---|---|---| | Initial skills | Human-written or pretrained | From zero | | Verifiers | Human-written / automatic | Self-constructed | | Supervision | Reference answers, human feedback | None | | Knowledge source | Fixed datasets | Public web resources | | Deployment | Tied to a specific model | Zero-shot cross-model transfer |

Three hard problems had to be solved: (1) where do verifiers come from without humans? (2) how to prevent the agent from accidentally finding benchmark answers online (leakage/cheating)? (3) how to make learned skills transferable across models without weight updates?

The Three-Stage Framework

Stage 1: Open-World Knowledge Acquisition

Given only an initial task prompt (no skills, verifiers, or answers), the agent searches public resources for:

• Task knowledge: docs, tutorials, code examples, Stack Overflow answers
• Verification anchors: independently checkable facts, specifications, syntax rules, and logical constraints that allow judging correctness without reference answers

A leak-detection mechanism filters retrieved content that resembles the target benchmark's test cases or answers (via semantic similarity, input-output matching, source credibility, and time-window checks), ensuring the agent never sees "exam questions" before final evaluation. Retrieved fragments are synthesized into a structured skill-building plan (task decomposition, tool choices, edge cases, verification strategy).

Stage 2: Leak-Free Skill Evolution

The agent writes its own tests: synthetic test cases are generated from verification anchors, with expected outputs derived from public specifications rather than leaked answers. It then runs an execute → diagnose → optimize loop until performance on the virtual test set is sufficient. Knowledge gaps (e.g., an unhandled edge case) automatically trigger targeted supplementary retrieval, closing the learning loop adaptively.

Stage 3: Zero-Shot Target Deployment

Evolved skills are packaged as structured Skill Cards (task description, execution steps, key parameters, validation rules, common pitfalls) and deployed via context augmentation — loaded into any target model's context at inference time. No weights are modified. Skills are model-agnostic: trained on GPT-4, they work directly with Claude 3.5, Llama-3-70B, and Qwen-72B. This also makes skills interpretable, editable, composable, and maintainable, unlike weight-encoded skills.

Experimental Results

• SkillsBench: +8.9% relative improvement over closed-world baselines that had reference answers, human verifiers, and pretrained skills — achieved under "blind exam" conditions.
• Verifier coverage: Self-constructed verifiers cover 88.9% of real test intent, meaning agent-generated practice tests align closely with human examiners' goals.
• Cross-model transfer: Skills evolved on GPT-4 transfer zero-shot to Claude 3.5, Llama-3-70B, and Qwen-72B with no adaptation; performance is retained or improved.
• Ablations: Removing knowledge acquisition collapses performance; removing leak filtering yields inflated scores with poor generalization; removing virtual test generation causes overfitting; replacing context-augmented deployment with fine-tuning destroys transferability. All three stages are necessary.

Why It Works

• Verification anchors shift the paradigm from "reference answers" to "grading criteria": syntactic specs, semantic constraints, logical consistency, and domain invariants let the agent judge whether output is *valid* without knowing the *correct answer*.
• Multi-dimensional leak detection (semantic similarity, input-output overlap, source credibility, temporal windows) blocks most unintentional cheating, though it is not 100% perfect.
• Context augmentation over weight editing trades minimal retrieval latency for transparency, portability, and composability of skills.

Industry Implications

• Agent framework developers should build in knowledge retrieval/filtering, externalized skill storage, and adaptive learning loops — rather than assuming humans provide tools and validators (as AutoGPT/LangChain/MetaGPT-style frameworks do).
• Model providers: capability need not depend on training exposure; a "skill marketplace" ecosystem (like an App Store for skills) becomes plausible, shifting competition toward context understanding and retrieval precision.
• Evaluation: benchmarks must distinguish true leakage (memorizing answers) from legitimate learning (distilling general knowledge from public sources) — a boundary that will keep blurring.
• Applications: rapid adaptation to new domains, e.g., a support agent learning from product docs, or code migration (Python 2 → 3) via retrieval plus self-generated validation rules.

Limitations and Open Questions

• Public resource dependency: niche domains with sparse/poor documentation block knowledge acquisition (a "digital divide").
• Anchor completeness: tasks lacking verifiable constraints (e.g., creative writing) are hard; OpenSkill suits programming, data processing, and format conversion best.
• Evolution overhead: multi-round retrieval and optimization can take minutes to hours; offline pre-evolution of a skill library is a practical mitigation.
• Safety: autonomous learning from public content risks absorbing harmful, biased, or outdated practices; safety filtering is an open requirement.

Conclusion

OpenSkill's significance is not the 8.9% benchmark gain but the redefinition of self-evolution's boundary conditions. It demonstrates that (1) agents can learn without reference answers by distilling verification anchors from public resources, (2) they can judge correctness without human verifiers using public specifications and logical constraints, and (3) skills can be reused across models via context-augmented, weight-free deployment. This marks a paradigm shift from "exam-taker" agents to "self-learners" — and a step from agents as tools toward agents as colleagues.

Reference: OpenSkill: Open-World Self-Evolution for LLM Agents, Zhiling Yan et al. — https://arxiv.org/abs/2606.06741