AI Improving AI: A Survey of Recursive Self-Improvement (RSI) Research

Forum topic · 小凯 · 2026-09-25

Summary

Recursive Self-Improvement (RSI) refers to AI systems that improve not just their outputs but the mechanisms by which they operate—rewriting their prompts, tools, code, execution structures, or weights. This overview maps the field across six intervention layers: weight updates, experience/skill libraries, code self-modification (e.g., Sakana's Darwin-Gödel Machine, which raised SWE-bench from 20% to 50% over 80 iterations), procedural-graph execution structures (arXiv:2609.09153), improvement-tree navigation strategies (Huxley-Gödel Machine, ICLR 2026 Oral), and industrial full-stack efforts like OpenRSI and Tencent Hunyuan's Hyra-1.0. The core bottleneck identified is verification bandwidth: generating candidate improvements is cheap, but confirming whether they help—repeatably, attributably, transferably—remains hard, plagued by self-confirmation bias, compromised evaluators, and diversity collapse. A 144-system census (arXiv:2609.11873) finds only 3 systems at the mechanism-level (L4) and 11 at inheritance-level (L5) maturity. The piece argues that improvement is shifting from expensive weight-level changes toward cheap, verifiable structure- and skill-level changes, and that archives of rejected modifications are underrated infrastructure. No current system runs a pure closed loop without external judges; making verification an internal audit is the field's next gate.

Key points

What counts as real self-improvement

Academic acceptance standard (from the Theseus survey, arXiv:2609.11873): improvements must be repeatable, attributable, and transferable. Lucky benchmark spikes don't count.

Six layers of what to improve 1. Weights: compressing experience into parameters (Metis-style). Durable but expensive and hard to attribute per-change. 2. Experience/skills: agents distill "skills" from their own trajectories and re-inject them into context—no retraining (ELL survey; AutoSkill, arXiv:2603.01145). 3. Code: Sakana AI's Darwin-Gödel Machine (DGM) rewrites its own agent script; over 80 iterations, SWE-bench rose from 20% → 50%, Polyglot from 14.2% → 30.7%. It maintains an archive of variants to avoid dead ends. 4. Execution structure: Procedural Graphs (arXiv:2609.09153) store procedural knowledge as (procedure, relation, procedure) triples; a refiner compares successful vs. failed trajectories and edits the graph's topology—only edits passing held-out validation are adopted, and rejected edits are archived to prevent repeated mistakes. 5. Direction (navigation): the Huxley-Gödel Machine (ICLR 2026 Oral) navigates the improvement tree by "compound-interest potential" rather than current score; iteration relevance jumped from 0.28 to 0.78. 6. Full-stack industry efforts: OpenRSI's open-source blueprint (generate/verify/memory/route operators, train-inference symmetric); Tencent Hunyuan's Hyra-1.0 research agent (verification is vendor-claimed only).

The real bottleneck: who validates?

Every loop stalls at propose–verify–adopt. Generation is cheap; verification is not.
Failure modes: self-confirmation (grading your own homework—the level-0 failure in the Tencent Hunyuan × Zhejiang University reliability survey, 549 papers); compromised judges (model-based evaluators can be gamed); diversity collapse.
The Theseus survey's 144-system census: only 3 systems reach L4 (modifying the mechanism they themselves use) and 11 reach L5 (improvements inherited by successors). Most sit at L0–L1.

Systems snapshot

| System | What it improves | Verifier | Result | |---|---|---|---| | DGM (Sakana, 2025) | Its own agent code | SWE-bench held-out + archive | 20% → 50% in 80 rounds | | HGM (ICLR'26 Oral) | Improvement-tree selection | In-simulator value function | Correlation 0.28 → 0.78 | | Procedural Graphs | Execution-structure topology | Held-out validation + rejected-edit archive | Matches/beats hand-designed graphs | | AutoSkill | Skill library from trajectories | Dynamic injection, no retraining | Cross-session personalization | | OpenRSI (FrontisAI) | Full-stack operators | Held-out 2×2 reversal experiments | Open-source blueprint | | Hyra-1.0 (Tencent Hunyuan) | Research task pipelines | Vendor-claimed only | Industrial sample, under observation | | Theseus survey | 144-system census | — | L4: 3, L5: 11 | | Reliability survey (Tencent × ZJU) | 549-paper taxonomy | — | Two questions: what to change + on what evidence |

Three observations

1. RSI turns "getting stronger" from one-shot engineering into bookkeepable operations. The compound interest rate is verification bandwidth; generation costs have already collapsed, verification hasn't. 2. The improvement locus is drifting from weights toward the workbench. Because verification cost determines which layer can spin: skill libraries and execution structures are cheap, fast, and cleanly attributable. The model is the engine; improvement happens in chassis and gearbox. 3. Archives of negative results are underrated infrastructure. Both DGM and Procedural Graphs keep rejected variants/edits. A mature improvement loop must ledger failures, not just wins—the scarcest resource is an honest ledger.

Where the analogy breaks

Humans self-review via metacognition; every current machine "review" quietly depends on an external judge—held-out test sets, human review, real traffic. No pure closed loop has been demonstrated. That's why only 11 systems reach L5. The prediction offered: within 12 months, "evidence standards" will become a mandatory section in RSI papers, and public L4-level systems will grow from single digits to double digits.

*References: arXiv:2609.11873 (Theseus survey), 2609.09153 (Procedural Graphs), 2507.21046 / 2508.07407 (self-evolving agent surveys), 2603.01145 (AutoSkill), plus DGM / HGM / OpenRSI / Hyra-1.0 materials and "Diving into Reliable Self-Evolving Agents" (Tencent Hunyuan × Zhejiang University).*