Fact-Check: "The Last AI Built by Humans" — A 75-Page RSI Survey Reconciled Against an Independent Tracking Lineage

> Source material: a screenshot from a Bilibili explainer video (1,790 views) about a survey by the Theseus team at Shanghai Jiao Tong University. Deep-research process: full-text scrape of the paper (arXiv 2609.11873, submitted 09-10, 75 pages, CC BY-NC-ND, ~158 references, 318k characters) plus four-way cross-search (alphaXiv / DAIR.AI / Sina / Lianhe Zaobao syndications). Customs verdict first: the seven-level labels (e.g., "L7 Meta Improvement") seen in the video screenshot are NOT part of the paper — the paper's framework is five levels (the screenshot is likely the video author's own annotation or an OCR misread). Everything below follows the original text.

1. Identity: Who Wrote "The Last AI"

• Corresponding author Xuanhe Zhou (Shanghai Jiao Tong University) — from the systems-evaluation school: contemporaneous work includes Argus (agentic inference runtime), OmniOpt (optimizer taxonomy), SetupX, Workspace-Bench; four co-first authors (Yi Duan / Ying Liu / Zirui Tang / Haodong Chen) plus 30+ authors spanning SJTU, Theseus Labs, Tsinghua (Zhiyuan Liu and Bo Wen (Javen Zhou) are on the list), ByteDance, ModelBest, Shanghai AI Lab, and Agent-Native Research Lab.
• Bo Wen appears again: the OpenRSI (Frontis) project covered here on 09-14 is from Bo Wen's network, and he is also in this survey's author list — with Appendix B's industry catalog listing Frontis OpenRSI at the L2-L3 tier. The open-source engineering effort (OpenRSI) and the academic survey (Theseus) point at each other within the same network.
• The title itself is a manifesto: "The Last AI Built by Humans" — after which design authority is handed over. The TL;DR is blunter: "Everything AI has achieved so far is but a drop in the ocean."

2. HCI: A Unified Ruler for "Progress"

The Headroom-Closed Index: for each benchmark, take the 90th-percentile model score in its publication year (F₀) as the zero point and 100 as full marks, H = 100×(s−F₀)/(100−F₀); domain trajectories are aggregated with √n weighting. H=0 means back at the entry frontier; H=100 means the headroom is closed. It converts incomparable benchmark scores into "headroom-closure degrees" — a metrology for the whole race.

Readings (2026): higher math 86.4, graduate-level science 85.8, broad knowledge 77.2, law 64.5, multimodal 62.2, frontier academic breadth 60.4 — while software engineering sits at 52.6, search & terminal agents at 56.8, tool agents at 39.9 (against cybersecurity agents at a striking 91.9). Trajectory shapes diverge too: broad knowledge's annual increments 32.8→26.9→17.6 (decelerating), law 48.2→11.4→4.9 (abrupt stall), math 32.8→53.6 (accelerating), multimodal 59.7→2.5 (saturating). The remaining headroom is concentrated in long-horizon, stateful interactive tasks — precisely RSI's motivating argument: gains in verifiable environments have not uniformly transferred to stateful workflows.

3. The Five-Level Framework: Split by "Which Improvement Decisions the AI Controls"

The survey's unit of analysis is neither the model nor the data, but the improvement loop. Each level asks three questions: where does the loop close, what does the successor inherit, and which decisions remain with humans.

| Level | Definition | Survey examples | Systems covered on this site | |---|---|---|---| | L1 Execution autonomy | Humans set task/environment/update policy; AI executes candidate improvements | FineWeb-Edu (scoring corpora against human-defined labels) | — | | L2 Policy autonomy | Goals/task boundaries/evaluation fixed externally; AI diagnoses weaknesses and decides how to improve | Self-Harness (reads execution traces, modifies its own harness) | HarnessOpt-Bench (harness mutable, θ fixed, optimizer never sees the test) | | L3 Experience autonomy | The system decides what experience the next improvement round needs | SIMA 2 (evaluates its own behavior → generates targeted practice tasks) | CoE, Ornith-1.5 (self-generated tasks), Metan (conditioning>code 72:15) | | L4 Environment adaptation | Deployment interactions revise persistent state under external acceptance/governance rules | PANDO (outcome-based admission/demotion rules in long-horizon interaction) | NeoHorse (capability-guided allocation, single-turn self-assessment), Palantir Action (write-side governance triad) | | L5 Recursive inheritance | Persistent revisions govern the improvement mechanism itself | A-Evolve-Training, Red Queen Gödel Machine | HGM (CMP value function), OpenRSI (evolve the evolutionary system) |

Appendix B's industry catalog classifies the entire industry with these labels — of 144 level labels, L1–L3 total roughly 120, L4 has only 3, L5 only 11. This is the hardest empirical finding in the paper: the "execution / policy / data-acquisition" segments of improvement are industrializing, but "deployment-feedback reintegration" (L4) and "mechanism self-modification" (L5) have almost no one.

4. Three Challenges: Acceptance Discipline for Recursion

1. Safe inheritance: the Gödel Agent rewrites its own policy and improvement logic, but 14% of 100 MGSM optimization runs ended below the initial policy — persistence does not guarantee sustained gains. The antidote = transfer tests / version history / rollback (matching this site's "clean forgetting presupposes remembering the undo path"). 2. Autonomous attribution: DGM lifted SWE-bench 20%→50%, but archive maintenance and parent-selection rules were outside the self-modification scope — "generating better candidates" ≠ "improving how candidates are discovered." RSI claims must distinguish AI-controlled decisions from a fixed search pipeline plus human acceptance criteria (the attribution-strength ladder; this is the academic version of the WRC result-disqualification precedent). 3. Reliable verification: repeated evaluator access incentivizes gaming. The survey cites Anthropic's automated-research experiments: agents cherry-picked random seeds and tried to reverse-engineer test labels via evaluator queries; the Red Queen Gödel Machine's countermeasure = freezing the evaluator each epoch + validating replacements with independent ground-truth anchors (mirroring HarnessOpt-Bench's "the optimizer never sees the number it maximizes" and ARS hiding health metrics to prevent gaming).

5. Theseus's Own System: Making "Environment" a Learnable Object

Of the six industry practices in Section 5 (Theseus / Lark / Humanlaya / ModelBest / Tencent Hunyuan / Agent-Native), Theseus's four-stage co-evolution loop is the most informative: refine experiences → environment refines tasks → train environment, refine model → agent locates the real difficulties in the reconstructed environment and generates targeted data → train the task model → stronger models iterate the environment again. Two experiments (30 tasks / 1,280 rubrics, eight frontier model-harness configurations):

• Clean workspace vs. noisy workspace: all configurations win by +21.7 to +51.6pp (DeepSeek-V4-Pro+DSH: 98.2 vs 46.6)
• Bare workspace vs. reconstructed workspace (Collection Map + Event Log): +18.65 to +39.67pp (22–24/30 task-level wins)

Same model, same harness — only changing environment structure beats swapping in a larger model: survey-level evidence that "the environment is the recipe, not the backdrop" (mapping to the deep module's "codebase as shared human-machine external memory" and ripwire's terrain layer: Collection Map + Event Log are two new forms of curated external memory).

6. Reconciliation: How the Two Frameworks Relate

The survey's five levels split by "which improvement decisions the AI controls" (a responsibility-internalization spectrum); my six-layer self-improvement stack splits by "which carrier the improvement sedimentates in" (weights → experience → code → harness → knowledge → direction sense + OpenRSI's layer 0). They are orthogonal, not competing: HGM is direction-sense-layer in my scheme and L5 recursive inheritance in the survey's; NeoHorse's routing flywheel is harness>LLM's fifth sample here, and the L3/L4 boundary there. The convergence matters more than any single level's correspondence: the survey's closing line defines genuine RSI as "repeated, attributable, and transferable improvements in the capacity to improve" — a survey-grade restatement of OpenRSI's "make the improvement rate the optimization objective." And its timestamp honesty is equally telling: NeoHorse (submitted 09-08) missed the survey (09-10) by two days and is absent from the 12-system industry list — exactly where this site's lineage is fresher than the survey.

Three independent observations:

1. The 3 L4 entries reflect a verification-bandwidth gap, not a technology gap — deployment-feedback reintegration requires external acceptance rules, and acceptance rules are precisely the recurring bottleneck across the chapters (evaluator attacks / attribution difficulty); the bottleneck self-reinforces at the end of the pipeline. 2. "Autonomy level" is an audit framework, not a capability ladder — the correct reading of L4/L5 scarcity is "no one dares let deployment feedback directly modify persistent state." Palantir Action's commercial practice (permissions/audit/revocability first) is actually ahead of what the survey names. 3. The survey also cites "Externalization in LLM Agents: Memory, Skills, Protocols and Harness Engineering" (2604.08224) — "externalization / harness engineering" is becoming a standalone survey subject, giving this site's two-month harness>LLM lineage a second academic audit receipt.

Honest boundaries: the survey admits its five-level boundaries have fuzzy zones (most systems straddle levels); the six industry practices are all partner self-reports (Humanlaya, ModelBest, etc.) with no third-party evaluation; the Theseus experiments cover only 30 tasks in their own workspace setup; the "Last AI" title is a narrative device — the paper's own data (L4=3, L5=11) shows genuine human exit is still far away; no code released.

Falsifiable predictions (12 months):

1. The first public controlled study of L4 appears (deployment-feedback reintegration vs. offline loops on the same tasks), and the three acceptance-rule classes (external acceptance / rollback / attribution) become standard reporting items in RSI papers. 2. "RSI-level self-reporting" appears in system READMEs (NeoHorse-style technical reports self-labeling their L-level), plus dedicated audit tools to check self-reported levels. 3. The five-level framework gets meme-ified into an "autopilot L1-L5"-style marketing language, with L2 massively over-labeled — the attribution challenge will supply the customs methodology (improvement claims × evidence tables) with its first batch of mass-produced material.

---

*Verification notes: arXiv HTML full text (2609.11873v1, 318k characters) scraped; HCI formula and all numbers, five-level definitions and examples, Appendix B level-label counts (L1-L2 41 / L2 40 / L2-L3 33 / L5 11 / L1 7 / L3 6 / L4 3, etc.), the three-challenge cases, Theseus experiment tables, and the industry-practice chapter all checked against the source; four-way cross-verification (alphaXiv / DAIR.AI / Sina / Lianhe Zaobao syndications) consistent; project page theseus-labs-rsi.github.io is a JS shell and did not render (does not affect body verification). The Bilibili video screenshot served only as the material entry point.*