Microsoft SkillOpt: One best_skill.md Travels Between Codex and Claude Code — and Scores Higher Than Home-Grown

Forum topic · 小凯 · 2026-08-09

Summary

SkillOpt, a joint effort from Microsoft, Shanghai Jiao Tong University, Tongji University, and Fudan University (arXiv 2605.23904, open-sourced on GitHub and PyPI), is a text-space optimizer that trains a natural-language skill document (a single Markdown file, 379–1,995 tokens) while keeping the target model frozen. An optimizer model reads scored rollouts, proposes bounded add/delete/replace edits, and only accepts changes that strictly improve a held-out selection split, exporting an auditable best_skill.md with a per-edit edit_apply_report.json. The headline result is cross-harness transfer: a skill optimized in Codex and loaded into Claude Code scored 81.8 on SpreadsheetBench versus 80.4 for a skill trained in Claude Code itself. Transfer is asymmetric, however — LiveMath reasoning-heavy skills retained only 10–30% of in-domain gains across harnesses, while spreadsheet 'workbook-level procedures' transferred well. Cross-model transfer within the GPT family is also uneven (82% to 16% retention). Training costs 0.6M–46.4M tokens per absolute test point, with zero inference overhead at deployment. Alongside protocol-level standards like Anthropic Agent Plugins 1.0.0, SkillOpt occupies the 'experience-layer portability' niche: skills that are portable, auditable, and reproducible across agent toolchains.

SkillOpt is a text-space optimizer from a Microsoft + Shanghai Jiao Tong + Tongji + Fudan collaboration. The paper is on arXiv (2605.23904), the code is open at microsoft/SkillOpt, and a skillopt package is on PyPI. It attacks the biggest real problem in the agent toolchain today: can an optimized skill document be moved across models and harnesses?

The answer is more aggressive than most expected: yes — and in one case, the transferred skill outperformed the destination harness's own trained skill.

What SkillOpt does

It trains a natural-language skill document (a Markdown file, 379–1,995 tokens); the target model stays frozen throughout.
An optimizer model reads scored rollouts and proposes bounded add/delete/replace edits; a held-out selection split accepts an edit only on strict score improvement.
The final artifact is a single best_skill.md, with every edit logged in edit_apply_report.json (per-edit accept/skip status). A human can read the whole skill in minutes and understand *why it works* — effectively returning the training product to an auditable, Git-diff-like form rather than fine-tuned weights or a prompt template.

Transfer experiments

The paper reports three groups of transfer experiments, each with a consistent three-column setup: baseline (no skill), direct (trained in-domain), transferred (skill moved in, zero additional optimization).

Cross-model (within GPT-5.4 family)

SpreadsheetBench, GPT-5.4-mini: baseline 36.1, direct 47.5, transferred 45.5 — 82% of in-domain gain retained (+9.4 of +11.4).
SpreadsheetBench, GPT-5.4-nano: only 16% retained (+3.0 of +19.0).
LiveMath, GPT-5.4-nano: transferred 28.8 actually exceeds direct 27.2 — explained as some learned procedures being irrelevant to the target model.
All rows stay within one GPT family; cross-family transfer (e.g., GPT → Qwen3.5) was not tested.

Cross-harness (same GPT-5.5) — the headline

SpreadsheetBench, Codex → Claude Code: baseline 22.1, direct 80.4, transferred 81.8 — the skill trained in Codex beat Claude Code's own skill by 1.4 points.
LiveMath, Codex → Claude Code: transferred 42.4 vs direct 56.5 (10% of gain retained); Claude Code → Codex: 48.0 vs 78.4 (30%). The asymmetry is the data point: procedural skills (inspect, verify, format) transfer; reasoning-heavy skills are bound to the harness that trained them.
The paper's explanation: SpreadsheetBench skills encode workbook-level procedures — inspect workbook structure and formulas first, then write evaluated static values instead of relying on Excel recalculation. These hold regardless of CLI tool. LiveMath skills bind to environment specifics (tool response formats, filesystem layout).

Cross-benchmark (same model)

OlympiadBench-trained skill moved to Omni-MATH: GPT-5.4 transferred 60.3 vs baseline 56.6 (+3.7); GPT-5.4-mini 36.6 vs 34.8 (+1.8); GPT-5.4-nano 40.1 vs 38.8 (+1.3). All positive — reusable math procedures do沉淀 — but small, and there is no direct column for comparison.

Why the artifact moves at all

All three execution modes (direct chat, Codex, Claude Code) consume the same best_skill.md format. Codex renders the skill as a per-task SKILL.md; Claude Code mirrors the same workspace contract via the claude CLI. Neither side gets a custom skill format — this shared contract is what makes cross-harness transfer possible.

Training cost

0.6M–46.4M training tokens per absolute test point (SpreadsheetBench 0.6M/point, DocVQA 46.4M/point). The optimizer runs only at training time; deployment adds zero inference cost. If a skill transfers across harnesses, the one-time cost amortizes across environments: optimize where it's cheapest, deploy where the product lives. The Codex → Claude Code 81.8 is the existence proof.

Position in the agent toolchain landscape

Anthropic Skills / Agent Plugins 1.0.0: schema + plugin.json standardization — protocol-layer portability.
Microsoft SkillOpt: text-space optimization + cross-model/cross-harness experience transfer — experience-layer portability.
Prime Agent / Continual Harness: harness CRUDs itself — adaptive-layer portability.

Together these form the full 2026 H2 picture: protocols make skills installable, experience makes them worth installing, adaptivity makes them evolve.

The underrated selling point: auditability

The deployed artifact is a text file a domain expert can read in minutes, with every edit traceable via edit_apply_report.json. Fine-tuned weights are unauditable black boxes by comparison. Portable + auditable is a fundamentally different operational posture — possibly SkillOpt's biggest selling point for enterprises, even though it isn't in the press title.

Sources (by authority)

Paper: https://arxiv.org/abs/2605.23904
GitHub: https://github.com/microsoft/SkillOpt
Project page: https://microsoft.github.io/SkillOpt/
Docs: https://github.com/microsoft/SkillOpt/blob/main/docs/index.md
PyPI: https://pypi.org/project/skillopt/
Demo video: https://youtu.be/JUBMDTCiM0M
MarkTechPost coverage: https://www.marktechpost.com/2026/08/05/microsoft-skillopt-agent-skill-transfer-portability
Baselines: GEPA (https://arxiv.org/abs/2507.19457), TextGrad (https://arxiv.org/abs/2406.07496), EvoSkill (https://arxiv.org/abs/2603.02766), Trace2Skill (https://arxiv.org/abs/2603.25158)
Benchmarks: SpreadsheetBench (https://arxiv.org/abs/2406.14991), LiveMathematicianBench (https://arxiv.org/abs/2604.01754), DocVQA (https://arxiv.org/abs/2007.00398)

Open questions

Cross-family transfer (GPT → Qwen, Claude → Llama) is untested.
The 81.8 figure is self-reported; external replication cost (0.6M tokens/point × rollbacks) is nontrivial and no public reproduction guide exists.
The root cause of LiveMath's 10% cross-harness retention (tool-call format vs prompt structure) isn't disentangled.
No official integration path combining SkillOpt training with Agent Plugins 1.0.0 distribution.
Anthropic Claude Skills appears nowhere as a comparison baseline — cooperation or competition is unclear.

One-line verdict: SkillOpt provides the first hard evidence in AI coding toolchains that a single best_skill.md can move between Codex and Claude Code — and score higher than home-grown. Harness competition is no longer just "whose harness is better" but also "whose skill can be carried away and still work," and SkillOpt is currently the only verifiable player on that axis.