Paper Overview
Field: ML Authors: Zisu Huang, Jingwen Xu, Yifan Yang Published: 2026-05-26 arXiv: 2505.21422
Summary
Language agents increasingly improve by reusing skills — structured procedural artifacts distilled from past experience. In particular, domain-level and model-generated skills are especially promising: they offer fast adaptation within a domain by encoding domain-specific recurring procedures, and they scale beyond labor-intensive hand-crafting. However, while extraction methods continue to proliferate, understanding remains limited, with no comprehensive study spanning the full skill lifecycle — experience generation, skill extraction, and skill consumption — to ask whether such skills actually work, when they work, and what makes them succeed or fail.
Key Findings
To close this gap, the authors build a utility-grounded evaluation framework providing systematic experimental results across extractors and consumer agents over five distinct agent task domains. Main results:
- Model-generated skills are beneficial on average, but exhibit non-trivial negative transfer.
- Behavior is inconsistent for both extractors and consumers: a model can be a strong extractor yet a weak consumer, or vice versa.
- Skill utility is not correlated with model scale or baseline task strength.
- How the composition of experience shapes skill quality.
- Which features characterize useful skills.
- How the same skill transfers across different consumer agents.
Analysis
The paper dissects each lifecycle stage to explain these patterns:
Meta-Skill
Finally, the findings are distilled into a concrete meta-skill that guides skill extraction toward features correlated with actual utility. This consistently improves skill quality across domains and substantially reduces negative transfer.
--- *Auto-collected on 2026-05-26*