The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents

Field: Machine Learning Authors: Darshan Tank, Baran Nama Published: 2026-07-24 arXiv: 2607.22520

Abstract (translated)

Adding procedural skills to an LLM agent is typically evaluated by average improvement in task success. However, this metric hides an important cost: skills can also make agents worse. The paper measures both sides by comparing agents with and without skills across nearly 6,000 runs spanning two office automation benchmarks and three model harness stacks. This distinguishes two outcomes: a regression is a task solved without skills but failed after skills are added; a residual failure is a task that fails both with and without skills.

Regressions turn out to be substantial enough that the best-performing skills outperform others primarily by regressing less, not by gaining more. The authors identify three causes of regression:

1. Skill description osmosis — a skill changes the agent's behavior merely by existing in context, even if never invoked. 2. Grounding displacement — the skill's prescribed process overrides how the agent interprets inputs. 3. Verification displacement — the procedure suppresses checks the agent would otherwise perform on its outputs.

Analysis of persistent failures reveals the same underlying pattern: existing skills overemphasize procedural guidance — the stage least responsible for failures — while underweighting grounding and verification, the main sources of remaining errors. After correcting evaluation artifacts and examining trajectories, many regressions and persistent failures can be recovered through better grounding and verification.

Takeaway: Procedural skills should be evaluated by decomposing their net effect into gains and regressions, not by aggregate improvement alone.

--- *Auto-collected on 2026-07-28*