Training Documents Instead of Models: How Microsoft's SkillOpt Turns Agent Skills into Trainable "External Parameters"
> Paper: arXiv:2605.23904 | GitHub: microsoft/SkillOpt | May 22, 2026
---
The Return of a Fundamental Question
Agent skills currently come from three sources: human-written, one-shot model generation, or model self-rewriting. Each has problems—handwriting is labor-intensive, one-shot generation lacks iterative feedback, and self-rewriting tends to degrade over time. Microsoft Research's SkillOpt pushes skill optimization from "craftsmanship" onto an engineering path with the same discipline as neural network training.
The core claim is simple: if a skill is an adaptation layer, it should be systematically trained like weights. Weights get backpropagation, learning rates, and validation-set gates—why shouldn't skills?
SkillOpt's answer: they can. And it all maps into text space.
---
The Training Loop: A Complete Mapping of Deep Learning Concepts into Text Space
| Deep learning | SkillOpt text-space counterpart | Role | |---------|---------------------|------| | Forward pass | Rollout on task minibatch | Collect scored trajectories | | Backpropagation | Reflection in natural language | Extract rules from error patterns | | Learning rate | Edit budget per epoch | Limit number and scope of edits | | Mini-batch | Separate reflection on success/failure subsets | Reduce reflection noise | | Momentum | Slow update + epoch-wise meta skill | Retain long-term knowledge | | Validation + early stopping | Held-out gate on candidate skill | Accept only genuine improvements | | Negative sampling | Rejected-edit buffer | Avoid repeating harmful edits |
This mapping is not a metaphor—it is a strict engineering implementation. Every step has explicit hyperparameters, is ablatable, and is reproducible.
---
The Four-Step Training Loop in Detail
Step 1: Rollout (forward)
The frozen target model executes tasks with the current skill, recording full trajectories—messages, tool calls, verifier feedback, final scores. Batch size controls the evidence per update: too small is noisy, too large is slow.
Step 2: Reflect (backward)
A separate optimizer model (typically stronger than the target) splits trajectories into success and failure groups and reflects on each in minibatches. The failure group proposes missing or corrected rules; the success group preserves existing effective behavior. Reflection produces structured add/delete/replace edit proposals.
The key here is minibatching—single trajectories tend to yield instance-specific fixes, while minibatches expose reusable procedural errors like "consistently searches the wrong source" or "writes answer in wrong format."
Step 3: Edit (parameter update)
The optimizer merges, deduplicates, and de-conflicts edit proposals, ranks them by expected utility, then truncates to the top Lt (textual learning rate). Lt=4 is the default; a cosine decay schedule moves from broad changes to fine adjustments.
Patch mode performs only local add/delete/replace—never a full skill rewrite. Protected slow-update fields cannot be overwritten by step-level edits, keeping fast and slow timelines parallel.
Step 4: Gate (validation)
A candidate skill must strictly improve the current best score on a held-out selection split to be accepted. Ties are rejected. This gate turns reflection into propose-and-test optimization rather than unconditional self-rewriting.
Rejected edits enter an epoch-local rejected-edit buffer; subsequent reflection calls can see "this direction was tried, lost X points, don't retry." Negative feedback is used only during training—zero overhead at deployment.
---
Epoch-wise Slow/Meta Update: Momentum in Text Space
Fast updates learn from the current batch; slow updates learn cross-epoch patterns. At each epoch's end, SkillOpt runs the same training samples with both skill versions and groups results as "improved," "regressed," "persistently failing," and "stably succeeding." The optimizer writes longitudinal guidance into a protected slow-update field—still gated before acceptance.
The meta skill lives on the optimizer side, summarizing which edit patterns worked, which were rejected, and which failures persist across epochs. This guidance is shown only to the optimizer and is not deployed with the skill. Deployment files stay compact (300–2,000 tokens) while training memory stays rich.
---
52/52: A Clean Sweep
Across 6 benchmarks × 7 target models × 3 execution harnesses = 52 evaluation cells, SkillOpt was best or tied-best in all of them.
| Target model | Direct Chat gain | Notes | |---------|----------------|------| | GPT-5.5 | +23.5 | Average over six benchmarks | | GPT-5.4 | +12.7 | — | | GPT-5.4-mini | +15.4 | — | | GPT-5.4-nano | +26.7 | Largest relative gain for small models | | GPT-5.2 | +16.6 | — | | Qwen3.5-4B | +19.2 | — | | Qwen3.6-35B-A3B | +9.1 | — |
Within-harness gains (GPT-5.5):
- Direct Chat: +23.5
- Codex loop: +24.8
- Claude Code: +19.1
- SearchQA: 77.7 → 87.3 (+9.6, near ceiling)
- SpreadsheetBench: 41.8 → 80.7 (+38.9, largest gain on procedural tasks)
- OfficeQA: 33.1 → 72.1 (+39.0)
- DocVQA: 78.8 → 91.2
- LiveMathematicianBench: 37.6 → 66.9
- ALFWorld: 83.6 → 95.5
- "Count any generic instance of the target receptacle as valid"
- "Maintain a tight numbered set of positions already explored and do not recheck them"
- "Broaden the search after several misses in the same type of position"
- Controllable: every edit is data-driven, validation-backed, traceable, and rollback-able
- Decoupled: skills live as Markdown files in Git, fully independent of the execution model
- Zero inference overhead: deployment loads only best_skill.md; the optimizer is never called
- Training isn't cheap: each round consumes target-model rollout tokens plus optimizer reflection tokens; the paper does not quantify full-cycle dollar cost
- Depends on grader quality: domains with hard-to-write reward functions (creativity, open-ended dialogue, tone) have weaker guarantees
- Optimizer threshold: the optimizer model must be strong enough for sensible reflection; only frontier models realize full potential
- Paper: arXiv:2605.23904 — Yang et al., "SkillOpt: Executive Strategy for Self-Evolving Agent Skills"
- GitHub: github.com/microsoft/SkillOpt (MIT License)
- Project page: microsoft.github.io/SkillOpt
- Third-party Codex implementation: github.com/SuperagenticAI/CodexOpt
Per-benchmark highlights (GPT-5.5 Direct Chat):
---
ALFWorld: A Four-Edit Evolution Example
On GPT-5.4-mini, ALFWorld went from 70.9% → 85.8% with only 4 accepted edits. Rules the skill learned on its own include:
These are the kinds of operational rules developers would take hours of debugging to distill—SkillOpt extracts them automatically from failure modes. Step 3 briefly regressed, but the slow-update mechanism recovered in the next step, with an overall monotonic upward trajectory—like SGD with momentum on a noisy loss.
---
Transfer: Train Once, Reuse Everywhere
Cross-model: The LiveMath skill trained on GPT-5.4 transferred to GPT-5.4-nano: +15.2. Small models gain procedural knowledge from a trained skill far faster than learning from scratch.
Cross-harness: The SpreadsheetBench skill trained in Codex transferred to Claude Code: +31.8. The same best_skill.md works across execution environments.
Cross-benchmark: The OlympiadBench skill also gave positive gains on Omni-MATH.
This means skills are not tied to their training environment—they are portable procedural memory. When the model changes, update the skill file; no fine-tuning required.
---
Ablations: Every Component Matters
| Removed component | SearchQA | SpreadsheetBench | LiveMath | |-----------|---------|------------------|----------| | No learning rate | -2.5 | -1.8 | -4.0 | | No rejected buffer | -1.6 | -4.6 | -2.4 | | No slow/meta update | -0.8 | -22.5 | -1.6 |
Removing the validation gate is the most damaging: drops of 2–22 points across benchmarks, because without a gate, noisy proposals degrade the skill over time. Slow/meta updates are especially critical for long procedural tasks like SpreadsheetBench—the 22.5-point drop shows momentum is a necessity, not a bonus, on long-horizon tasks.
---
Industrial Perspective: A Fourth Path
Improving agent quality has traditionally meant one of three routes: 1. Handwriting ever-longer prompts (unscalable) 2. RLHF/SFT weight fine-tuning (expensive, model-bound) 3. Shipping a new system prompt and hoping (blind, fragile)
SkillOpt opens a fourth path:
CodexOpt (a third-party implementation) has already brought this paradigm into the Codex ecosystem: uv run codexopt improve --live starts SkillOpt-style optimization.
---
Honest Limitations
---
References
> The core judgment of this post: SkillOpt is not an incremental improvement on prompt engineering—it redefines the skill optimization paradigm, treating natural-language documents as external parameters of a frozen model and training them with full deep-learning discipline. The 52/52 result shows this is not a toy idea but systematic improvement.