SkillGrad: Optimizing Agent Skills Like Gradient Descent
In one sentence: A Penn State team proposes SkillGrad, which treats an agent skill package as an optimizable parameter and iterates on it like gradient descent. Failed trajectories provide the loss signal, success trajectories provide protective signals, and a textual momentum mechanism accumulates recurring diagnostic patterns across iterations. On SpreadsheetBench, GPT-5.4 improves from 55.8% (Base Skill) to 71.1%, outperforming the strongest training baseline Trace2Skill by 3 pp; GPT-4.1 is even more dramatic, going from 36.7% to 54.2%, leading Trace2Skill by 17 pp. Ten iterations cost about $6.40—no model fine-tuning required, only the skill files change.
1. The Skill File Problem: Generated, But Not Good Enough
LLM agent "skills" are a lightweight way to adapt to a domain—reusable procedural knowledge stored as structured files (SKILL.md) that the agent loads when encountering matching tasks. No model fine-tuning needed; swapping the skill set switches domains.
But the problems are obvious:
- LLM-generated skills are far lower quality than expert-written ones
- Third-party downloaded skills may be outdated, incomplete, or buggy
- Existing evolution methods (EvoSkill, Trace2Skill) rely on heuristic reflection with no explicit optimization framework
- L2: 40 lines → ~120 lines at iteration 1 → stabilizes at ~154 lines (mean)
- L3: 0 → grows monotonically to ~13,000 characters (iteration 10)
- Failed trajectories (currently failing): corrective diagnosis—"what went wrong, how to fix it"
- Contrastive success trajectories (initially failed → currently succeeding): protective diagnosis—"what changed from failure to success? This change is worth keeping"
- \(M_t\) persistent pattern memory: records mechanisms recurring across iterations. Each pattern has: anchor, appearance iterations, description, current best execution action, fix history.
- \(O_t\) current-batch overlay: compact update signals for each batch task.
- Iteration 1 (bootstrap): ~1,170 words added, ~80 removed
- Iterations 2–10 (incremental): ~330 words/iteration (240–410 band), removals consistently low
- SpreadsheetBench Verified: in-domain (spreadsheet operations), 40 train / 20 val / 120 test. Hard accuracy: all output cells must match.
- WikiTableQuestions (WikiTQ): out-of-domain (semi-structured table QA), fixed sample of 70. Denotation accuracy against gold answers.
- No Skill: raw LLM, no skill files
- Base Skill: initial skill (LLM-generated or third-party)
- Trace2Skill (Ni et al., 2026): trace-to-skill distillation
- EvoSkill (Alzubi et al., 2026): failure-driven evolution
- GPT-5.4: 55.83% → 71.11% (+15.28 pp), beating Trace2Skill and EvoSkill by 3.05 pp each
- GPT-4.1: 36.67% → 54.17% (+17.50 pp), leading Trace2Skill and EvoSkill by 16.95 pp
- WikiTQ (out-of-domain): GPT-5.4 82.38% vs Trace2Skill 79.05% (+3.33 pp); GPT-4.1 73.65% vs 60.00% (+13.65 pp)
- Early: feature engineering—hand-designed features
- Middle: gradient descent—automatically optimized parameters
- Now: prompt engineering—hand-tuned prompts
- SkillGrad: prompt optimization—automatically iterated skill files
- Gradient descent optimizes numerical parameters (weight matrices)
- SkillGrad optimizes textual parameters (structured skill files)
- Gradient descent's loss is differentiable
- SkillGrad's loss is discrete (success/failure)
- Paper: SkillGrad: Optimizing Agent Skills Like Gradient Descent
- arXiv: 2605.27760
- Authors: Hanyu Wang, Yifan Lan, Bochuan Cao, Lu Lin, Jinghui Chen
- Institution: Pennsylvania State University
- Date: 2026-05-26
- Core concepts: skill packages as parameters, gradient descent analogy, trajectory-level loss evidence, contrastive success diagnostics, textual momentum, layer-aware patching (L1/L2/L3)
- Limitations: mostly spreadsheet tasks, non-monotonic textual updates with no convergence guarantee, no formal theory
- Code: not explicitly stated as open-sourced in the paper, but the framework should be reproducible
EvoSkill is "failure-driven"—fix what failed. Trace2Skill "distills"—extract local lessons from execution traces and hierarchically integrate them. Both are empirical: no optimization objective, no convergence guarantees, no mechanism to prevent over-optimization.
SkillGrad's angle: treat the skill package as parameters, execution traces as loss evidence, and the whole iteration process as gradient descent.
2. Layered Skill Parameters: L1/L2/L3 Architecture
| Layer | Content | Loading strategy | |-------|---------|------------------| | L1 Metadata | YAML description (name, description, activation conditions) | Routing only, decides when to activate | | L2 SKILL.md | Core principles, workflows, operating modes, common pitfalls | Always loaded, kept compact | | L3 Resources | Detailed examples, edge cases, condition-triggered procedures | Conditionally loaded, read on demand |
L2 must be compact—every line consumes the context budget of every future run. L3 can grow freely—no cost when unused. The patcher must decide: does new knowledge go into L2 (important principles) or L3 (edge cases)?
Size evolution over iterations:
> L3 snowballs, but L2 converges—core knowledge is compressed into the always-loaded head, edge knowledge settles into the on-demand tail.
3. Loss Evidence: Not Just Failures, But Failure-to-Success Contrasts
Binary success/failure loss is too coarse: success gives no signal; failure just triggers a fix. SkillGrad uses trajectory-level loss:
Two evidence types:
Training tasks are sampled from the initial skill's failure pool, not randomly—so at iteration 1, many tasks fail initially but may now succeed, making contrastive success trajectories naturally abundant.
> Key insight: even when a task currently succeeds, contrasting the success with a nearby failure still yields useful improvement signal, solving the sparsity of purely binary loss.
4. Textual Momentum: Accumulating Recurring Diagnostic Patterns
4.1 The Gradient Descent Analogy
| Gradient descent | SkillGrad textual momentum | |------------------|---------------------------| | \(v_t = \gamma v_{t-1} + g_t\) | \((M_t, O_t) = \text{Momentum}(M_{t-1}, D_t, S_t)\) |
4.2 Dual-State Structure
4.3 Dynamics
| Metric | Value | |--------|-------| | Cumulative pattern saturation | Iteration 7 (~10 patterns) | | New pattern rate, iteration 1 | ~3 per iteration | | New pattern rate, iterations 5–9 | 0–1 per iteration | | Active patterns (iteration 3+) | Stabilizes at 4–5 |
Late in training, new patterns nearly stop appearing—the skill has converged to a stable set of knowledge patterns. That's momentum's role: stable experience is retained, unstable one-off fluctuations are filtered out.
5. Layer-Aware Patching: Turning Diagnoses into Edits
The patcher doesn't just append diagnoses to the skill file. It must:
1. Pattern-level generalization: when multiple diagnoses point to the same mechanism, produce a single generalized edit rather than a task-level fix list
2. Layer-aware routing: decide whether content goes into L2 (always loaded) or L3 (conditional)
3. Anchor system: kebab-case slugs (e.g., inspect-before-edit, mapping-shapes) stably locate sections, preventing duplicates
Patch magnitude:
Ablation: without momentum, L2 word count inflates 29.7% (2,416 vs 1,863) while L3 shrinks 32%—more content is forced into L2, weakening conditional loading.
6. Experiments: The Numbers
6.1 Datasets
6.2 Baselines
6.3 LLM-Generated Skill Initialization
| Method | GPT-5.4 SpreadsheetBench | GPT-5.4 WikiTQ | GPT-4.1 SpreadsheetBench | GPT-4.1 WikiTQ | |--------|--------------------------|----------------|--------------------------|----------------| | No Skill | 62.50 | 78.57 | 44.17 | 52.86 | | Base Skill | 55.83 | 77.14 | 36.67 | 48.58 | | Trace2Skill | 65.28±2.93 | 79.05±1.35 | 37.22±3.54 | 60.00±5.83 | | EvoSkill | 68.06±0.48 | 78.09±0.67 | 37.22±2.58 | 53.33±7.50 | | SkillGrad | 71.11±1.73 | 82.38±1.78 | 54.17±3.54 | 73.65±2.50 |
Notable numbers:
GPT-4.1 gains more than GPT-5.4—SkillGrad helps weaker models more, since they depend more heavily on skill guidance.
6.4 Third-Party Skill Initialization
| Method | GPT-5.4 SpreadsheetBench | GPT-5.4 WikiTQ | GPT-4.1 SpreadsheetBench | GPT-4.1 WikiTQ | |--------|--------------------------|----------------|--------------------------|----------------| | Base Skill | 60.00 | 78.57 | 33.33 | 42.86 | | Trace2Skill | 63.89±0.48 | 81.91±1.78 | 38.89±3.85 | 51.43±1.17 | | EvoSkill | 63.61±1.92 | 80.95±3.56 | 36.94±1.92 | 43.33±4.86 | | SkillGrad | 69.44±0.48 | 83.34±0.67 | 45.83±3.66 | 53.81±2.43 |
Third-party initial skills are typically better than LLM-generated ones (Base Skill 60% vs 55.8%), but SkillGrad's improvement remains stable—69.44% vs 63.89% (+5.55 pp).
6.5 Ablations
| Variant | Accuracy | Change | |---------|----------|--------| | Full SkillGrad | 72.50% | — | | No momentum | 65.83% | -6.67 pp | | Failure-only diagnostics | 68.33% | -4.17 pp |
Momentum matters more than contrastive success diagnostics—removing momentum costs 6.67 pp vs 4.17 pp for contrastive success. Cross-iteration stability is worth more than single-iteration signal richness.
6.6 Hyperparameters and Cost
| Batch size | Training trajectories | Accuracy | |-----------|----------------------|----------| | 2 | 20 | 70.0% | | 4 (default) | 40 | 72.5% | | 6 | 60 | 70.8% |
Bigger batches aren't better—4 is the sweet spot; 6 declines (likely more noise).
| Iteration | Accuracy | |-----------|----------| | 1 | 63.3% | | 4 | 65.8% | | 7 | 67.5% | | 10 (default) | 72.5% | | 13 | 70.0% |
Iteration 10 is the best checkpoint; performance drops by 13—overfitting or non-monotonic updates. Unlike numerical gradients, textual updates have no convergence guarantees.
Cost: a 10-iteration run costs about $6.40 ± 0.38. Per-iteration cost grows from ~$0.35 (iteration 1) to ~$0.85 (iteration 10) as the momentum and patching prompts accumulate pattern state.
7. Takeaway: From "Tuning Prompts" to "Optimizing Parameters"
SkillGrad's core contribution is not a specific trick but a perspective shift—treating skill files not as hand-written documents but as optimizable parameters.
This mirrors machine learning's evolution:
The difference from gradient descent:
SkillGrad uses the LLM as a "differentiable bridge"—its diagnostic ability converts discrete signals into textual gradients, and its editing ability converts textual gradients into parameter updates. Essentially, it uses the LLM's soft-computation capability to emulate a numerical optimization loop.