SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
> Paper: SkillCraft: Can LLM Agents Learn to Use Tools Skillfully? > Authors: Shiqi Chen, Jingze Gai, Ruochen Zhou, Jinghan Zhang, Tongyao Zhu, et al. (University of Oxford, City University of Hong Kong, HKUST, Northwestern University, NUS) > arXiv: https://arxiv.org/abs/2603.00718 > Code: https://github.com/shiqichen17/SkillCraft > Project page: https://skillcraft-website.github.io/page
Why do AI assistants act like forgetful interns?
Current tool-using agents can call APIs, but they restart from scratch every time: re-deriving the same tool-call sequences, mis-ordering steps on complex tasks, or looping on a single tool until tokens run out. SkillCraft targets this gap: true intelligence is not about using tools, but about abstracting complex tool-calling flows into reusable skills.
The SkillCraft benchmark
Two-axis difficulty
The benchmark provides 126 real-API tasks across six domains (coding, finance, entertainment, etc.), with difficulty scaled on two dimensions:
| Dimension | Description | Example | |---|---|---| | Quantitative (volume) | Number of atomic tools involved | 3 → 8 → 15 tools | | Structural (logic) | Dependencies and composition between tools | linear chain → parallel branches → nested conditions → loops |
Four native skill-management tools
| Tool | Function | |---|---| | Save Skill | Package a tool-call chain into a reusable code block | | Query Skill | Retrieve matching skills from the library | | Read Source | Inspect a skill's implementation before reuse | | Execute Skill | Call a stored skill directly for a subtask |
These form a closed loop: complete tasks → extract patterns → store → retrieve on similar tasks.
Lightweight protocol: automatic skill acquisition and reuse
1. Skill Acquisition: the system tracks tool-call traces, identifies frequent stable sub-sequences, generates parameterized code, validates it by execution, and only then stores it — avoiding a polluted library without any human-written scripts. 2. Skill Reuse: on new tasks, the agent queries the library first, executes matching skills, and falls back to standard tool calls otherwise; new experience is deposited again.
Results: up to 80% token reduction
| Metric | Without skills | With skills | |---|---|---| | Token usage | baseline | up to 80% lower | | Success rate on hard tasks | baseline | substantially higher | | Response time | long (full re-inference) | short (direct skill calls) |
Cross-model findings
- Skill mode universally improves efficiency across tested models (including GPT-5.2, Claude, DeepSeek).
- But skill quality varies widely: good skills transfer across models, poor skills fail everywhere. Skill acquisition acts as a magnifying glass on a model's underlying induction ability.
- ToolLLM: large-scale tool learning, no skill abstraction/reuse measurement
- LATM / CREATOR: create new tools rather than compose existing ones
- Voyager: skill libraries, but in Minecraft rather than real APIs
- MemSkill: memory mechanisms, not skill quality evaluation
- Skill granularity is set by predefined rules; may be suboptimal
- Limited domain coverage (126 tasks, six domains)
- Skill conflicts unresolved when multiple skills match
- Long-horizon degradation of composed shallow skills unknown
- Chen, S., Gai, J., Zhou, R., Zhang, J., Zhu, T., et al. (2026). SkillCraft: Can LLM Agents Learn to Use Tools Skillfully? *arXiv preprint* arXiv:2603.00718.
- Code: https://github.com/shiqichen17/SkillCraft
- Project page: https://skillcraft-website.github.io/page
Counter-intuitive finding
> Deeply nested skills cause cascading errors; shallow, refined skills are the best value.
Once a low-level skill has a bug, every skill above it breaks. Shallow skills wrapping 2–4 atomic tools, composed explicitly rather than nested, are more robust — echoing "composition over inheritance" in software engineering.
Evaluation paradigm shift
The paper states: *"Success rate strongly correlates with tool composition ability at test time."*
| Old paradigm | New paradigm | |---|---| | Single-call accuracy | Cross-task reuse | | Static tool sets | Dynamic skill libraries | | Instance-level success | Skill-level abstraction | | "Can it use tools?" | "Can it accumulate skills?" |
Relation to prior work
Limitations and future directions
Future work: adaptive granularity, skill versioning, cross-agent skill sharing, and a decentralized skill marketplace.
Conclusion
SkillCraft is less a new benchmark than a paradigm statement: the next stage of AI tool use is not "more tools" but "better skills" — compositional skill acquisition as a core capability. An agent with verified, reusable skill combinations evolves from a tool user into a senior architect.