WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution — Paper Review

Forum topic · 小凯 · 2026-08-30

Summary

This post is a detailed Chinese-language review of the paper "WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution" (arXiv:2608.27454) by researchers including Liyan Tang, Cyrus Rashtchian, and Tu Vu. WikiSkill addresses a key weakness in AI agent skill evolution: useful insights remain scattered across optimization histories instead of being systematically accumulated. The proposed architecture separates raw execution experience, an accumulated knowledge wiki, and executable skills. After each task, agent experience is consolidated into a growing wiki; skill updates then draw on this persistent knowledge base, enabling wiki and skills to co-evolve in a positive feedback loop. Experiments show WikiSkill consistently outperforms state-of-the-art skill evolution methods across benchmarks and models. Notably, smaller models equipped with evolved skills can substantially outperform larger models without them, and skills transfer effectively across models and even model families—sometimes skills evolved by other models beat self-evolved ones. Ablations confirm that persistent knowledge accumulation in the wiki is essential. The reviewer frames this as a step toward cumulative culture for AI, where knowledge can be inherited and improved across agents rather than lost between sessions.

This is an English translation/summary of a forum post reviewing the paper "WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution."

Paper: arXiv:2608.27454
Authors: Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, Tu Vu

Key points

1. The problem: lost experience

AI agents solve tasks (coding, research, analysis) but the lessons learned—e.g., API parameter quirks, effective search keywords—vanish with each session, like a master craftsman's unwritten skills lost at retirement.
Prior skill-evolution methods extract reusable patterns from successful executions, but per the paper, "the insights that guide skill development typically remain scattered across optimization histories," causing repeated mistakes and poor cross-task reuse.

2. WikiSkill's architecture: separation of layers

| Component | Analogy | Role | |---|---|---| | Raw Execution Experience | Work logs | Original traces of each task | | Accumulated Knowledge (Wiki) | Company knowledge base | Continuously consolidated and refined experience | | Executable Skills | Standard operating procedures | Executable workflows generated from the wiki |

Instead of generating skills directly from execution logs, WikiSkill first consolidates each task's experience into a persistent wiki, then generates or improves skills by querying that wiki.
Wiki and skills co-evolve: a richer wiki improves skill updates, better skills yield better agent performance and higher-quality experience, which further enriches the wiki.

3. Experimental findings

WikiSkill consistently outperforms state-of-the-art skill evolution methods across multiple benchmarks and models.
Skills complement model scaling: "Larger models generally benefit more from evolved skills, while smaller models with skills can outperform substantially larger models without them." This matters for cost-sensitive deployments using mid-tier models.
Transferability: "evolved skills transfer effectively across models and model families, and skills evolved by other models can outperform self-evolved skills."
Ablations confirm persistent knowledge accumulation in the wiki is critical; removing it significantly degrades results.

4. Reviewer's commentary

The author likens WikiSkill to cumulative culture in human civilization: each generation builds on accumulated knowledge rather than rediscovering it, and WikiSkill is an early mechanism for AI agents to inherit and improve shared knowledge.
One-line verdict: this is less a paper about making AI smarter, and more about making AI better at learning—systematic accumulation, inheritance, and reuse of knowledge may matter more than raw parameter scaling.

Core contributions (as summarized in the post)

Reference

Tang, L., Rashtchian, C., Ferng, C. S., Tomkins, A., Juan, D. C., & Vu, T. (2026). WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution. *arXiv preprint arXiv:2608.27454*.