Paper Overview
Field: Machine Learning Authors: Yuval Kansal, Karen Ullrich, Polina Kirichenko, Ilia Kulikov, Niraj K. Jha, Mohammad Pezeshki, Mark Ibrahim Published: 2026-10-27 arXiv: 2610.31846
Summary
Intelligent agents should learn from their mistakes so that experience leads to continuous improvement. Yet today's agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities.
The authors introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary in-context retries and policy updates. When an agent fails a task, it reflects on its own trajectory to extract a lesson and immediately distills it into a lightweight low-rank adapter. By keeping the original weights frozen, Self-Direction prevents adapter training from overwriting base-model parameters, while training only ~0.02% of the base model's parameter count per adapter. The agent then stores these adapters in a persistent lesson library, automatically routing them to solve related follow-up problems.
Key Results
Experiments with agents built on Qwen-series open-source models across open-ended research (GAIA), tool-use workflows (Claw-Eval), and terminal tasks (Terminal-Bench 2.1) show that Self-Direction:
- Improves overall success rates by 8.9 to 17.0 percentage points, remaining competitive with frontier systems
- Reduces repeated mistakes on future tasks, nearly doubling the reliability of corrections
- Enables transfer of learned lessons: a few adapters generalize to dozens of new tasks with no additional training
Original Abstract (excerpt)
> Intelligent agents should learn from their mistakes so that experience leads to continuous improvement. Yet today's agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities. We introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary in-context retries and policy updates. When an agent fails a task, it reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter...
Paper: arXiv 2610.31846