> arXiv 2609.24972 (2026-09-21) | Google Cloud AI Research + UNC + Stanford + WashU | github.com/google-research/rrsi (open-sourced, 350 stars) | The ninth post in this blog's RSI series, and the first "defensive" one.
Positioning in the series
Eight prior posts covered harness self-improvement: OpenRSI, NeoHorse, Theseus, Dream-RSI, HGM, Metan, Agensh, plus Tencent Hunyuan's reliability survey. They all asked the same question: can the harness be improved? RRSI asks a different one: are the improvements real? This is the first paper to make overfitting in harness evolution its main subject. Defensive in orientation, but possibly more practical than all eight predecessors.
1. The disease: overfitting in self-improvement
Current harness evolution follows the same recipe: an LLM proposer makes iterative component-level edits to the harness, and whichever edit raises the benchmark score is kept. The paper calls this agent-system-level RSI.
The problem lies in the scoring scheme. The evolve set is a finite benchmark that gets queried repeatedly by every round of evolution. The paper traces this directly to the roots of adaptive data analysis (Dwork et al. 2015): adaptively re-testing hypotheses against the same dataset leads to overfitting as a statistical inevitability, not an engineering oversight. Each evaluation round is another adaptive look at a finite evolve set; apparent gains may just be benchmark-specific fitting, evaluation noise, or unnecessary complexity growth.
This is not just a theoretical concern. Cited evidence: gains reported by existing harness evolution methods fail to survive on a different benchmark suite. The paper's own experiments are harsher — four recent methods (Meta-Harness, AHE, TTHE, HarnessX) were run from the same baseline with the same candidate budget. All improved on the evolve set, but out-of-distribution rankings fully inverted: top evolve-scorer Meta-Harness gained only 0.9 out of distribution, HarnessX ended flat, and AHE and TTHE landed below where they started, with TTHE regressing 1.7 points. Only RRSI's OOD average (43.6 vs 39.7) beat baseline by more than a point. This inversion table is the sharpest thing in the paper.
2. The medicine: constrain the search, not the edit space
RRSI's design stance is clean: the harness edit space stays fully open (prompts, control flow, configs, context management, tools, skills, memory, sub-agents are all editable); what gets regularized is the search trajectory. The proposer side controls "where search capacity is spent"; the selection side controls "what qualifies as permanent state." Seven mechanisms, each with an ML regularization counterpart:
Proposer side, three:
- Annealed edit budget (L0-style): the number of edits a candidate can bundle per round is cosine-annealed from many to few — early rounds allow batched probes of new mechanisms; later edits grow sparser, making attribution easier.
- Evidence-aware credit assignment: hypotheses, diffs, scores, and cost changes are logged throughout. Rejected mechanisms become negative evidence; successful ones get explicit credit — the search stops re-spending on falsified hypotheses.
- Structured exploration: when progress retreats into the noise band, search stalls, and a small share of budget is forcibly allocated to never-touched components — an entropy regularization analogue.
- Leakage screening (critic): before evaluation, a critic reads the diff and rejects edits that encode task names, entity names, task-specific values, or answers — the key is "before evaluation": leaked candidates never get inflated scores, so they have no downstream appeal.
- Stability-aware acceptance: before evolving, the untouched baseline is repeatedly evaluated to estimate an empirical noise band δ; candidate scores must clear an S*−δ floor — preventing the search from degrading along a downhill slope "too small to look like anything but noise."
- L2-style cost gate: when a score gain exceeds the noise band, cost growth must satisfy ΔC ≤ β₀ + β₁·ΔS — inference overhead can only rise in exchange for measurable performance gains.
- L1-style pruning: components without strictly positive gains over a fixed window are flagged for deletion. The paper's own words: "mechanisms must continuously earn their position, rather than squatting in corners that pure score-based evolution has no deletion incentive to clean up."
Selection side, four:
3. The numbers
Eight benchmarks across three domains, with the policy frozen at Claude Opus 4.8 throughout. Evolve gains were deliberately kept small: Terminal-Bench +6.0, EngDesign +4.9, Harvey LAB only +1.1. In exchange, held-out performance never regressed: SWE-bench Verified +1.8 (repo-level fixes never scored during evolution), Harvey in-distribution held-out +2.3, three out-of-distribution agentic benchmarks +3.5 to +4.7, and Frontier-Eng +4.3 Medal (a relative +24.3%). EngDesign scores are deterministically produced by a frozen simulator, closing off the "pander to the judge" path — the gains held.
Swapping backbones, the story repeats. With Gemini 3.5 Flash as the search policy: Terminal-Bench 64.6→78.7 (+14.1, the biggest number in the abstract), SWE-bench still +2.2. Even stronger is cross-model transfer: a harness searched by 3.5 Flash, installed unchanged on Gemini 3.1 Flash Lite — a weaker model never involved in the search — moved 11.2→14.6. The mechanisms don't bind to the capability level that found them: a harness is a program, not weights, and reusability now has hard evidence.
The ablation table says it plainly: strip out all regularization and the evolve score hits the highest in the field (92.8), but OOD drops to 40.3 — within a point of the un-evolved baseline — at 3.80M tokens per trial, 1.6x RRSI's cost. Unconstrained selection rules spent most accepted edits on noise and context bloat, not mechanisms. The paper is candid about the other side: all evolved harnesses are more expensive than the un-evolved H0 (1.56M tokens) — evolution does buy some gains with test-time compute; budget determines how much.
4. Boundaries and connections
Three limitations stated plainly: no weight-layer changes (frozen backbone); still dependent on a finite evolve set and several regularization hyperparameters; longer-horizon self-improvement unverified.
Connecting back to this blog's main line, the paper lands in three places. First, the economics of validation bandwidth gets formalized at paper level: a finite evolve set means finite validation bandwidth. This blog has argued at the governance layer that "a validation budget is fundamentally an effective sample size"; RRSI connects that intuition to the mathematical roots of Dwork's adaptive data analysis — repeatedly querying the same validation set has a quantifiable statistical cost. Second, the annealed budget directly responds to the DGM attribution problem flagged in the Theseus survey: bundled edits can't be attributed, and tightening the budget over rounds makes attribution progressively easier. Third, the evaluate-then-transfer protocol (evolve gains don't count; no OOD regression does) is the assertion-strength ladder applied to harness evolution methods — the prediction, planted in a survey post nine days ago, that "evidence would become a mandatory field," is being fulfilled by top-venue-grade papers.
A falsifiable prediction: within 12 months, harness evolution papers will make "out-of-distribution retention" a standard acceptance criterion, and evolve-set-only gain tables won't make the main table — just as agent papers reporting only in-context gains have gradually stopped being taken seriously. Another bet: RRSI's regularization suite will be split into reusable components (the leakage critic and the noise floor look most like standalone products), plugged into the OpenRSI open-source roadmap.
---
*Sources: arXiv 2609.24972v1 full text (including ablations and appendix pointers), github.com/google-research/rrsi (created 9/16, 350 stars), Rujun Han's LinkedIn announcement. All figures are as reported in the paper; the four baseline methods were re-run under identical conditions by the paper's authors, with no third-party replication.*