Training Documents, Not Models: Microsoft SkillOpt Turns Agent Skills into Trainable External Parameters
> Paper: arXiv:2605.23904 | GitHub: microsoft/SkillOpt | May 22, 2026
The Return of a Fundamental Question
Agent skills currently come from three sources: human-written, one-shot model-generated, or model self-rewritten. Each has problems — hand-writing burns people, one-shot generation lacks feedback loops, and self-rewriting tends to degrade over iterations. SkillOpt, proposed by Microsoft Research, pushes skill optimization from "craftsmanship" onto an engineered path with the same discipline as neural network training.
The core claim is simple: if a skill is an adaptation layer, it should be systematically trained like weights. Weights have backpropagation, learning rates, and validation gates — why shouldn't skills?
SkillOpt's answer: they can. And it all maps cleanly into text space.
The Training Loop: A Full Mapping of Deep Learning Concepts into Text Space
| Deep learning | SkillOpt text-space counterpart | Purpose | |---|---|---| | Forward pass | Rollout on task minibatch | Collect scored trajectories | | Backpropagation | Reflection in natural language | Extract rules from error patterns | | Learning rate | Edit budget per epoch | Limit number and scope of edits | | Mini-batch | Separate reflection on success/failure subsets | Reduce reflection noise | | Momentum | Slow update + epoch-wise meta skill | Retain long-term knowledge | | Validation + early stopping | Held-out gate on candidate skill | Accept only real improvements | | Negative sampling | Rejected-edit buffer | Avoid repeating harmful edits |
This mapping is not a metaphor — it is a strict engineering implementation. Every step has explicit hyperparameters, is ablatable, and is reproducible.
The Four-Step Training Loop in Detail
Step 1: Rollout (forward)
The frozen target model executes tasks with the current skill, recording full trajectories — messages, tool calls, validator feedback, final scores. Batch size controls the evidence per update: too small is noisy, too large is slow.
Step 2: Reflect (backward)
A separate optimizer model (typically stronger than the target) splits trajectories into success and failure groups and reflects on each as minibatches. The failure group proposes missing or corrected rules; the success group preserves already-effective behavior. Reflection yields structured add/delete/replace edit proposals.
The key here is minibatching — a single trajectory tends to produce case-specific fixes, while minibatches expose reusable procedural errors such as "consistently searches the wrong source" or "writes answer in wrong format."
Step 3: Edit (parameter update)
The optimizer merges, deduplicates, and de-conflicts edit proposals, ranks them by expected utility, and truncates to the top Lt (textual learning rate). The default is Lt=4, with a cosine decay schedule moving from large rewrites to small refinements.
Patch mode only performs local add/delete/replace — never a full skill rewrite. Protected slow-update fields are not overwritten by step-level edits, keeping the fast and slow timelines running in parallel.
Step 4: Gate (validation)
A candidate skill must strictly improve the current best score on a held-out selection split to be accepted. Ties are rejected. This gate turns reflection into propose-and-test optimization rather than unconditional self-rewriting.
Rejected edits enter an epoch-local rejected-edit buffer, so subsequent reflection calls can see "this direction was tried, lost X points, don't repeat it." Negative feedback is used only during training — zero overhead at deployment.
Epoch-wise Slow/Meta Update: Momentum in Text Space
Fast updates learn from the current batch; slow updates learn cross-epoch regularities. At each epoch boundary, SkillOpt runs the same training samples with the before and after skill versions, grouping results as "improved," "regressed," "persistently failing," and "stably succeeding." The optimizer writes longitudinal guidance from this and places it in the protected slow-update field — still subject to the gate.
The meta skill lives on the optimizer side, summarizing which edit patterns worked, which were rejected, and which failures persist across epochs. This guidance is only shown to the optimizer and is not deployed with the skill. Deployment files stay compact (300–2000 tokens) while training memory stays rich.
52/52: A Clean Sweep
6 benchmarks × 7 target models × 3 execution harnesses = 52 evaluation cells; SkillOpt is best or tied-best in all of them.
| Target model | Direct Chat gain | Notes | |---|---|---| | GPT-5.5 | +23.5 | Average across six benchmarks | | GPT-5.4 | +12.7 | — | | GPT-5.4-mini | +15.4 | — | | GPT-5.4-nano | +26.7 | Largest relative gain for small models | | GPT-5.2 | +16.6 | — | | Qwen3.5-4B | +19.2 | — | | Qwen3.6-35B-A3B | +9.1 | — |
Within-harness gains (GPT-5.5):
- Direct Chat: +23.5
- Codex loop: +24.8
- Claude Code: +19.1
- SearchQA: 77.7 → 87.3 (+9.6, near ceiling)
- SpreadsheetBench: 41.8 → 80.7 (+38.9, largest gain on procedural tasks)
- OfficeQA: 33.1 → 72.1 (+39.0)
- DocVQA: 78.8 → 91.2
- LiveMathematicianBench: 37.6 → 66.9
- ALFWorld: 83.6 → 95.5
- "Count any generic instance of the target receptacle as valid"
- "Maintain a tight numbered set of positions already explored and do not recheck them"
- "Broaden the search after several misses in the same type of position"
- Controllable: every edit is data-driven, validation-backed, traceable, and rollback-able
- Decoupled: skills live as Markdown files in Git, fully independent of the execution model
- Zero inference overhead: deployment loads only best_skill.md; the optimizer is never called
- Training cost is not trivial: each rollout consumes target-model tokens plus optimizer reflection tokens; the paper does not quantify the full dollar cost of a training cycle
- Depends on grader quality: guarantees are weaker in domains where reward functions are hard to write (creativity, open-ended dialogue, tone)
- Optimizer threshold: the optimizer model must be strong enough to produce sensible reflection; frontier models are needed to realize full potential
- Paper: arXiv:2605.23904 — Yang et al., "SkillOpt: Executive Strategy for Self-Evolving Agent Skills"
- GitHub: github.com/microsoft/SkillOpt (MIT License)
- Project page: microsoft.github.io/SkillOpt
- Third-party Codex implementation: github.com/SuperagenticAI/CodexOpt
Per-benchmark highlights (GPT-5.5 Direct Chat):
ALFWorld: A Four-Step Evolution Example
On GPT-5.4-mini, ALFWorld improved from 70.9% to 85.8% using only 4 accepted edits. Rules the skill learned by itself include:
These are operational rules that would take a developer hours of debugging to distill — SkillOpt extracted them automatically from failure patterns. Step 3 briefly regressed; the slow-update mechanism recovered it the next step, with an overall monotonic upward trajectory — much like SGD with momentum on a noisy loss.
Transfer: Train Once, Reuse Everywhere
Cross-model: The LiveMath skill trained on GPT-5.4 transferred to GPT-5.4-nano for +15.2. Small models gain procedural knowledge from trained skills far faster than learning from scratch.
Cross-harness: The SpreadsheetBench skill trained in the Codex loop transferred to Claude Code for +31.8. The same best_skill.md works across execution environments.
Cross-benchmark: The OlympiadBench skill also yields positive gains on Omni-MATH.
This means skills are not bound to their training environment — they are portable procedural memory. When the model changes, you update the skill file instead of fine-tuning.
Ablations: Every Component Matters
| Removed component | SearchQA | SpreadsheetBench | LiveMath | |---|---|---|---| | No learning rate | -2.5 | -1.8 | -4.0 | | No rejected buffer | -1.6 | -4.6 | -2.4 | | No slow/meta update | -0.8 | -22.5 | -1.6 |
Removing the validation gate is the most damaging: drops of 2–22 points across benchmarks, because without the gate, noisy proposals degrade the skill over the medium-to-long term. Slow/meta update is especially critical for long procedural tasks like SpreadsheetBench — a 22.5-point drop shows that momentum is not a nice-to-have on long-horizon tasks; it is a necessity.
The Industry Perspective: A Fourth Path
Improving agent quality traditionally has three routes: 1. Hand-writing ever-longer prompts (not scalable) 2. RLHF/SFT weight fine-tuning (expensive, model-locked) 3. Shipping new system prompt versions and hoping (blind, fragile)
SkillOpt opens a fourth path:
CodexOpt (a third-party implementation) has already brought this paradigm into the Codex ecosystem: uv run codexopt improve --live starts SkillOpt-style optimization.