Training Documents, Not Models: How Microsoft's SkillOpt Turns Agent Skills into Trainable External Parameters
> Paper: arXiv:2605.23904 | GitHub: microsoft/SkillOpt | May 22, 2026
A Fundamental Question Returns
Agent skills currently come from three sources: hand-written by humans, generated one-shot by a model, or self-rewritten by a model. Each has problems—hand-writing is labor-intensive, one-shot generation lacks feedback loops, and self-rewriting tends to degrade. Microsoft Research's SkillOpt moves skill optimization from "craft" to an engineering discipline comparable to neural network training.
The core claim is simple: if a skill is an adaptation layer, it should be systematically trained like weights. Weights have backpropagation, learning rates, and validation gates—why shouldn't skills?
SkillOpt's answer: they should. And everything maps cleanly into text space.
The Training Loop: A Complete Mapping of Deep Learning Concepts into Text Space
| Deep learning | SkillOpt text-space counterpart | Role | |---|---|---| | Forward pass | Rollout on task minibatch | Collect scored trajectories | | Backpropagation | Reflection in natural language | Extract rules from error patterns | | Learning rate | Edit budget per epoch | Limit number/scope of modifications | | Mini-batch | Separate reflection on success/failure subsets | Reduce reflection noise | | Momentum | Slow update + epoch-wise meta skill | Preserve long-term knowledge | | Validation + early stopping | Held-out gate on candidate skill | Accept only real improvements | | Negative sampling | Rejected-edit buffer | Avoid repeating harmful edits |
This mapping is not metaphorical—it is a strict engineering implementation. Every step has explicit hyperparameters, is ablatable, and is reproducible.
The Four-Step Training Loop in Detail
Step 1: Rollout (forward)
The frozen target model executes tasks with the current skill, recording full trajectories—messages, tool calls, verifier feedback, and final scores. Batch size controls the evidence available per update: too small is noisy, too large is slow.
Step 2: Reflect (backward)
A separate optimizer model (typically stronger than the target) splits trajectories into success and failure groups and reflects on each in minibatches. The failure group proposes missing or corrected rules; the success group preserves behaviors that already work. Reflection outputs structured add/delete/replace edit proposals.
Minibatching is key here—single trajectories produce instance-specific fixes, while minibatches expose reusable procedural errors like "consistently searches the wrong source" or "writes answer in wrong format."
Step 3: Edit (parameter update)
The optimizer merges, deduplicates, and resolves contradictions among edits, ranks them by expected utility, then truncates to the top Lt (the textual learning rate). The default is Lt=4, with a cosine decay schedule moving from large rewrites to fine adjustments.
Patch mode only makes local edits rather than rewriting the entire skill. Protected slow-update fields cannot be overwritten by step-level edits, keeping fast and slow timelines running in parallel.
Step 4: Gate (validation)
A candidate skill must strictly improve the current best score on a held-out selection split to be accepted. Ties are rejected. This gate turns reflection into propose-and-test optimization rather than unconditional self-rewriting.
Rejected edits enter an epoch-local rejected-edit buffer, so later reflection calls can see "this direction was tried, lost X points, don't go there again." Negative feedback is used only during training—zero overhead at deployment.
Epoch-wise Slow/Meta Update: Momentum in Text Space
Fast updates learn from the current batch; slow updates learn cross-epoch patterns. At each epoch's end, SkillOpt runs the same training samples with both the previous and current skill versions, grouping outcomes into "improved," "regressed," "persistently failing," and "stably succeeding." The optimizer writes longitudinal guidance from this and places it in a protected slow-update field—still gated before acceptance.
The meta skill lives on the optimizer side, summarizing which edit patterns worked, which were rejected, and which failures persist across epochs. This guidance is shown only to the optimizer and is never deployed with the skill. Deployment files stay compact (300-2000 tokens) while training memory stays rich.
52/52: A Clean Sweep
6 benchmarks × 7 target models × 3 execution harnesses = 52 evaluation cells, and SkillOpt was best or tied-best in all of them.
| Target model | Direct Chat gain | Notes | |---|---|---| | GPT-5.5 | +23.5 | Average across six benchmarks | | GPT-5.4 | +12.7 | — | | GPT-5.4-mini | +15.4 | — | | GPT-5.4-nano | +26.7 | Largest relative gain among small models | | GPT-5.2 | +16.6 | — | | Qwen3.5-4B | +19.2 | — | | Qwen3.6-35B-A3B | +9.1 | — |
Within-harness gains (GPT-5.5):
- Direct Chat: +23.5
- Codex loop: +24.8
- Claude Code: +19.1
- SearchQA: 77.7 → 87.3 (+9.6, near ceiling)
- SpreadsheetBench: 41.8 → 80.7 (+38.9, largest gain on procedural tasks)
- OfficeQA: 33.1 → 72.1 (+39.0)
- DocVQA: 78.8 → 91.2
- LiveMathematicianBench: 37.6 → 66.9
- ALFWorld: 83.6 → 95.5
- "Count any generic instance of the target receptacle as valid"
- "Maintain a tight numbered set of positions already explored and do not recheck them"
- "Broaden the search after several misses in the same type of position"
- Controllable: every edit is data-driven, validated, traceable, and rollback-able
- Decoupled: skills live as Markdown files in Git, fully independent of the execution model
- Zero inference overhead: deployment loads only best_skill.md; the optimizer is never called
- Training is not cheap: each round consumes target-model rollout tokens plus optimizer reflection tokens; the paper does not quantify full-cycle dollar cost
- Depends on grader quality: domains where reward functions are hard to write (creativity, open-ended dialogue, tone) have weaker guarantees
- Optimizer threshold: the optimizer model must be strong enough for sensible reflection; frontier models unlock the full potential
- Paper: arXiv:2605.23904 — Yang et al., "SkillOpt: Executive Strategy for Self-Evolving Agent Skills"
- GitHub: github.com/microsoft/SkillOpt (MIT License)
- Project page: microsoft.github.io/SkillOpt
- Third-party Codex implementation: github.com/SuperagenticAI/CodexOpt
Per-benchmark highlights (GPT-5.5 Direct Chat):
ALFWorld: A Four-Step Evolution Example
On GPT-5.4-mini, ALFWorld improved from 70.9% to 85.8% with only 4 accepted edits. Rules the skill learned on its own include:
These are the kinds of operational rules developers spend hours debugging to summarize—SkillOpt distilled them automatically from failure patterns. Step 3 briefly regressed, and the slow-update mechanism recovered it the next step, with an overall monotonic trajectory—like SGD with momentum on a noisy loss.
Transfer: Train Once, Reuse Everywhere
Cross-model: The LiveMath skill trained on GPT-5.4 transferred to GPT-5.4-nano, +15.2. Small models gain procedural knowledge from a trained skill far faster than learning from scratch.
Cross-harness: The SpreadsheetBench skill trained under Codex transferred to Claude Code, +31.8. The same best_skill.md works across different execution environments.
Cross-benchmark: The OlympiadBench skill also gave positive gains on Omni-MATH.
This means skills are not tied to their training environment—they are portable procedural memory. When the model changes, just update the skill file; no fine-tuning needed.
Ablations: Every Component Matters
| Removed component | SearchQA | SpreadsheetBench | LiveMath | |---|---|---|---| | No learning rate | -2.5 | -1.8 | -4.0 | | No rejected buffer | -1.6 | -4.6 | -2.4 | | No slow/meta update | -0.8 | -22.5 | -1.6 |
Removing the validation gate was the most damaging: drops of 2-22 points across benchmarks, since without a gate, noisy proposals degrade the skill over time. Slow/meta updates are especially critical for long procedural tasks like SpreadsheetBench—the 22.5-point drop shows momentum is not a luxury but a necessity for long-horizon tasks.
Industry Perspective: A Fourth Path
Historically, three routes existed to improve agent quality: 1. Hand-writing ever-longer prompts (not scalable) 2. RLHF/SFT fine-tuning of weights (expensive, model-bound) 3. Shipping a new system prompt and hoping for the best (blind, fragile)
SkillOpt opens a fourth path:
CodexOpt (a third-party implementation) has brought this paradigm into the Codex ecosystem: uv run codexopt improve --live starts SkillOpt-style optimization.
Honest Limitations
References
> The core judgment of this article: SkillOpt is not an incremental improvement on prompt engineering—it redefines the skill optimization paradigm by treating natural-language documents as external parameters of a frozen model, trained with the full discipline of deep learning. The 52/52 result shows this is not a toy idea but a systematic improvement.