RSI-Opt: Recursive Self-Improvement Optimizer Agents for Model Training

Paper Overview

Field: ML Authors: Binghang Lu, Yige Sun, Guang Lin Published: 2026-10-27 arXiv: 2610.32026

Summary

Conventional neural-network training typically follows a predefined optimization recipe, which motivated the creation of LLM agents that monitor training dynamics and adjust optimizer settings as training progresses. However, a fixed agent policy does not accumulate experience across runs.

The authors introduce RSI-Opt, a plug-and-play Recursive Self-Improvement Optimizer Agent that evolves the knowledge guiding this process:

• a prompt
• a playbook of procedures
• a library of tested optimizer code
• a decision cadence

A meta-agent rewrites this policy using evidence from earlier runs, and an archive retains evaluated policies for further evolution.

Results

Across 5 generations of evolution on three training tasks, the meta-agents developed reusable procedures and code and conducted their own CPU proxy experiments.

Key results:

• RSI-Opt substantially improved optimizer-agent performance; the representative raw-space summary z-score increased from +10.48 to +14.33 on the Opus 5.5 evolution suite.
• Through cross-model policy transfer, the score improved from -0.36 to +3.11 on GPT-5.5 held-out tasks.
• On the Burgers PDE, RSI-Opt reduced average relative L2 error by roughly 2.7x with Opus 5.5 and 8.8x with GPT-6.1 versus the fixed-prompt agent baseline, although PDE transfer varied across LLMs and tasks.

Controlled comparisons indicate that evolved policies combine reusable optimization knowledge with per-task adaptation: initial decisions capture most of the observed gains on seen tasks, while continued adaptation improves results on unseen tasks.

Conclusion

These findings demonstrate that accumulating process knowledge and executable tools across training runs improves optimizer-agent performance and supports transfer to scientific machine learning.

--- *Auto-collected on 2026-10-27*