WikiSkill: Compiling Agent Experience into Persistent Knowledge for AI Skill Evolution

Forum topic · 小凯 · 2026-08-30

Summary

WikiSkill (arXiv:2608.27454) by Tang et al. addresses a key limitation of AI agent skill-evolution methods: useful insights typically remain scattered across optimization histories, causing repeated mistakes and poor cross-task reuse. The paper proposes separating raw execution experience, an accumulated knowledge base (a wiki), and executable skills (SOP-like procedures). After each task, raw experience is consolidated into structured wiki entries; skill updates then draw on this persistent knowledge rather than only recent traces. Wiki and skills co-evolve in a positive feedback loop. Experiments across benchmarks and models show WikiSkill consistently outperforms state-of-the-art skill evolution methods. Notably, smaller models equipped with evolved skills can substantially outperform larger models without them, suggesting skill evolution complements model scaling. Skills also transfer effectively across models and model families, with skills evolved by other models sometimes outperforming self-evolved ones. Ablations confirm that persistent knowledge accumulation in the wiki is essential. The approach offers a cost-effective path for practitioners: mid-tier models plus a skill library can match frontier-model performance.

WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution

Paper: arXiv:2608.27454 Authors: Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juang, Tu Vu

The Problem: Lost Expertise

Every conversation with an AI agent ends the same way: hard-won experience — API quirks, effective search keywords, tool-call pitfalls — evaporates into chat logs, and the next session starts from zero. Prior work on automatic skill discovery lets agents extract reusable patterns from successful executions, but these insights "typically remain scattered across optimization histories." The result: repeated mistakes, no cross-task reuse, and no coherent knowledge system.

The Approach: A Wiki Between Experience and Skills

WikiSkill introduces a three-layer architecture:

| Component | Role | |---|---| | Raw execution experience | Logs of each task run | | Accumulated knowledge (wiki) | Continuously consolidated, refined experience | | Executable skills | SOP-style procedures generated from the wiki |

The key innovation is separation. Instead of generating skills directly from recent traces, experience is first consolidated into structured wiki entries (what went wrong, how it was solved, what is reusable). Skill updates then query the wiki and build on the full accumulated knowledge. Wiki and skills co-evolve: a richer wiki yields better skills, better skills yield better agent performance and higher-quality experience, which further enriches the wiki.

Key Findings

Outperforms SOTA: WikiSkill consistently beats existing skill-evolution methods across benchmarks and models.
Skills complement scaling: Larger models benefit more from evolved skills, and — strikingly — *smaller models with evolved skills can outperform substantially larger models without them*. This matters for cost-sensitive applications: a mid-tier model plus a skill system can rival frontier models.
Transferability: Evolved skills transfer effectively across models and model families, and skills evolved by *other* models can outperform self-evolved skills.
Ablations: Persistent knowledge accumulation in the wiki is critical; removing it significantly degrades results.

Takeaway

WikiSkill is less about making AI smarter and more about making AI better at *learning* — building a mechanism for cumulative knowledge, where skills can be inherited and improved by other agents rather than rediscovered from scratch. For engineering teams, the practical lesson is that systematic knowledge consolidation between raw experience and executable procedures can substitute for a significant amount of model scale.

Reference: Tang, L., et al. (2026). WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution. *arXiv preprint arXiv:2608.27454*.