RSI-Anything: Breaking the Chicken-and-Egg Problem of Recursive Self-Improvement with World Models

Forum topic · ✨步子哥 · 2026-10-12

Summary

RSI-Anything, a framework from CUHK and Stanford researchers, addresses the core challenge of recursive self-improvement (RSI): how can an AI system convert interaction experience into lasting self-improvement without expensive external judges? The key obstacle is a coupled-update chicken-and-egg problem—improving the policy shifts the candidate distribution, invalidating the world model's predictions, and vice versa. RSI-Anything solves this with prediction-observation pairing built on a JEPA-based predictive architecture with orthogonal prediction factorization, plus asynchronous updates and compatibility checks that gate which component versions are retained. Empirical results span five domains: code repair success on CodeContests rose from 26.0% to 31.2% under the same online-testing budget; MiniWoB++ checkbox task success jumped from 51% to 99% with 39.4% fewer actions; lab planning improved from 21.80% to 46.01% across five policy versions; and Suzuki-Miyaura reaction optimization beat Bayesian optimization by up to 13.99 percentage points. Code and the LabHorizon benchmark are open-sourced on GitHub.

Human chess players improve by reviewing their games—but most AI systems stop evolving once training ends. Recursive self-improvement (RSI) aims to solve this: converting interaction experience into persistent self-improvement without costly external judges. A recent paper from CUHK and Stanford, RSI-Anything, offers an elegant answer: let AI act as its own judge—using a world model to predict action outcomes, using predictions to guide learning, and correcting the predictions themselves from observations.

The Core Problem: A Chicken-and-Egg Dilemma

RSI is fundamentally a loop: *improve the system with experience → the improved system generates new experience → improve again*. The hidden trap: you want to learn from feedback, but executing candidate actions is costly (API credits, lab reagents, GUI time). A simulator could rank candidates—but updating the policy changes the candidate distribution, degrading the world model's predictions, while updating the world model changes which learning data gets selected. The two components cannot be improved independently.

The Solution: Prediction-Observation Pairing + Asynchronous Updates

The framework builds on JEPA-Anything's predictive architecture (latent-space prediction with orthogonal prediction factorization, OPF) and runs a four-step loop:

1. Prediction-guided selection: the world model scores candidate actions; the best is executed 2. Execution for feedback: only the selected candidate runs, yielding real observations 3. Feedback-driven updates: observations both improve the policy/workflow and assess world-model reliability 4. Compatibility-checked retention: component-level and pair-level checks decide which versions to keep

The key innovation is asynchronous updating: policy and world model evolve on different timescales—a reliable predictor supports multiple policy-improvement rounds, and new evidence triggers predictor adaptation independently.

Results Across Four Domains

Code repair (CodeContests, 200 problems): random selection 26.0% → static verifier 26.3% → RSI-Anything with fixed world model 29.2% (candidate-ranking accuracy 66.5% → 72.8%) → with world-model adaptation 31.2%, all under the same testing budget (3.72 tests/problem). Also 46.37% on APPS Introductory and 33.05% on LiveCodeBench v6.
GUI interaction (MiniWoB++ checkbox): 51% → 99% success rate; actions per episode cut 39.4% (5.30 → 3.21); 237 redundant clicks and 99 erroneous submissions reduced to 0 and 1. On OSWorld: +18.4% over the initial agent, +5.5% over RLAnything.
Lab planning (LabHorizon, 200 Level-2 tasks): five consecutive policy versions climbed steadily: 21.80% → 38.03% → 41.99% → 44.56% → 46.01%, with no plateau; the final 4B model matched the Actor-Simulator-Selector reference (45.32%).
Chemical reaction optimization (Suzuki-Miyaura): GC screening (20 experiments) 93.0% vs Bayesian optimization 87.5% (+5.5pp); flow LC-MS screening (40 experiments) 96.15% vs 82.16% (+13.99pp).
Immunology (end-to-end): Agent Genesis workflow search + an immunology world model produced a forward-looking therapeutic hypothesis: SIRPα inhibition + IL-36γ axis modulation.

Engineering Insights

1. Prediction-observation pairs are the universal currency: one experience ("I predicted A, observed B") serves both candidate evaluation and predictor reliability assessment. 2. Asynchronous updates reduce risk: components improve at their own pace rather than moving in lockstep. 3. Compatibility checks as gatekeepers: pair-level checks ensure the new policy and new model are genuinely better together—preventing the "better policy, worse model" trap. 4. Domain generality from shared mechanisms: the same select-update-retain loop works across code, GUI, lab planning, chemistry, and immunology; only state encoding, action space, and feedback sources differ.

How This Differs from Prior RSI Work

Prior RI falls into two lines: code self-improvement (CURE, Darwin Gödel Machine, AIDE2) and environment adaptation (Voyager, RLAnything). RSI-Anything's essential distinction is explicitly introducing a world model as predictor and treating prediction reliability as a first-class citizen—addressing the neglected question of whether the evaluator is still "of the same era" as the system it evaluates.