*Editor's note: Structured summary of a long Chinese forum post. All claims, dates, and figures below are as stated in the source post.*

MUSE-Autoskill: The Toolbox That Writes Its Own Diary

The post opens with a metaphor: an old carpenter's toolbox carries decades of tacit knowledge—how each plane behaves, when it needs extra sharpening. Today's LLM agents get "skills" (reusable capability packages, e.g., Voyager's code library, Anthropic's SKILL.md format), but existing methods treat skills as one-shot, static, isolated artifacts. MUSE-Autoskill (ByteDance, arXiv:2605.27366, 30 pages) proposes a framework where agents "write their own diaries, run their own checkups, and replace their own parts."

Key points

Four gaps in existing skill systems

• Creation–usage mismatch: skills are generated offline, divorced from the real execution environment. MUSE makes skill_create an in-loop skill invoked inside the agent's ReAct cycle.
• No per-skill memory: global episodic buffers are too coarse. MUSE gives every skill a .memory.md file recording failure modes, format quirks, and performance traps, loaded with SKILL.md on each use.
• Static, unvalidated skills: MUSE enforces a unit-test gate—each skill ships a tests/ directory and must pass sandboxed tests (with update_skill patch loops) before registration.
• Poor context handling: adaptive compression over a DAG of conversation nodes (Level-1 node summarization, Level-2 segment merging, head/tail preserved), plus persistent session snapshots for resumable long-horizon tasks.

Five-stage lifecycle

Creation → Evaluation → Memory → Management → Refinement. Management includes retrieval via SKILL.md metadata (progressive disclosure), refinement from failures, merging of overlapping skills, and pruning of dead ones. Generated skills are longer than human-written ones (median 326 vs. 146 lines) but replace verbose trial-and-error reasoning.

Results on SkillsBench (51 tasks, GPT-5.5 backbone, 5 runs per config)

• Human skills help all agents: Codex 52.11% → 67.28% (+15.17pp), Hermes 47.89% → 61.21% (+13.33pp), MUSE 53.19% → 68.40% (+15.21pp).
• MUSE-Autoskill ranks first in 3 of 4 super-domains (Data Analysis, Document Processing, Ops & Planning) and overall; it trails Codex in Science & Engineering (72.86% vs. 78.57%), attributed to three boundary cases where verifiers penalized unspecified methodological choices.
• Self-generated skills: succeeded on 35/51 tasks (68.6%); on those tasks, accuracy hit 87.94%, above the 68.40% human-skill ceiling. Standouts: flink-query 20% → 100%, weighted-gdp-calc 20% → 100%, adaptive-cruise-control 40% → 100%. Overall 51-task score of 60.35% is dragged down by the 16 failed-generation tasks counted as 0%.
• Cross-agent transfer: injecting MUSE-generated skills into Hermes yields +10.51pp (47.89% → 58.40%), closing 79% of the gap to Hermes+human skills; MUSE and Hermes with identical generated skills differ by only 1.95pp. The post calls this the only empirically validated cross-agent skill transfer in the literature.
• Efficiency: generated skills give MUSE −20% tokens, −37% latency, 19→15 turns; Hermes −48% tokens, −30% latency. A one-time 383K-token generation pays back after ~3 reuses; the generated-skill configuration is the only Pareto-optimal point on reward-latency and reward-tokens axes.

Honest limitations (as listed in the paper)

• Coverage bottleneck: 16/51 tasks failed at Phase 1—the agent couldn't solve them at all, so no skill could be distilled. This is a base-agent-capability issue.
• Single-trajectory bias: each skill is distilled from one success trajectory. Example: hvac-control regressed from 80% to 20% because a calibration procedure fit to one run's noise profile broke elsewhere.
• Statistical power: 5 runs per task gives wide confidence intervals on binary rewards.
• Same-task coupling: skills are generated and evaluated on the same task, a potential source of the high 87.94% figure; cross-task transfer tests are needed.

Closing thought

The post ends by arguing MUSE makes the carpenter's tacit, non-transferable tool-wisdom *replicable and portable*: per-skill .memory.md files are each tool's "usage autobiography." Quoting the paper—"This work provides a scalable approach toward agents with continuously evolving capabilities"—the author reframes it: this is a story about agents learning how to learn.

References (from the post)

• Paper: MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation — Huawei Lin, Peng Li, Jie Song, Fuxin Jiang, Tieying Zhang (ByteDance / RIT), 30 pages, 8 figures, 13 tables
• Link: https://arxiv.org/abs/2605.27366 (arXiv:2605.27366 [cs.AI]), released May 26, 2026
• Benchmark: SkillsBench (51 tasks); backbone: GPT-5.5; baselines: Codex, Hermes; related: Anthropic Agent Skills, Voyager, AutoSkill, EvoSkill, SkillGen, SkillOS, MemGPT, Reflexion, ReAct