On September 28, a 14-page working paper appeared on arXiv with a question as its title: *"Will AI R&D automation trigger an intelligence explosion?"* It carries 22 signatures.
The author list is itself news. Hinton—holder of both the Nobel Prize in Physics and the Turing Award; Bengio and Andrew Barto, both Turing Award winners; Jakub Pachocki, OpenAI's chief scientist; Jack Clark, Anthropic co-founder; Eric Horvitz, Microsoft's chief scientific officer; plus DeepMind's Thore Graepel and Princeton's Jeff Clune. Pachocki's company and Clark's company are fighting for the same market. The project was led by the University of Cambridge's CASP programme (Cambridge Programme on AI Science & Policy), spearheaded by Alan Chan and Sören Mindermann. Hinton promoted it on X in early October.
These people usually disagree considerably. Their ability to co-sign a single warning is more noteworthy than the warning itself.
What the paper says
First, the mechanism. The intelligence explosion in this paper is not mysticism—it is a multiplication: more and better AI systems doing AI research → better research produces better AI → better AI expands the research workforce. The paper calls this a recursive feedback loop.
The mechanism rests on numbers already happening. Anthropic officially reports that AI-written code passing human review rose from low single digits in January 2025 to over 80% by May 2026. Another Anthropic measurement report says autonomously completed R&D work—under only high-level human supervision—was 1% in March of this year and 26% by August: a 26-fold increase in five months. OpenAI and Google have each made statements as well; the former's code-executing agents are already used across training, evaluation, and protecting future models.
There is also METR's famous metric: the length of tasks AI can complete used to double every 7 months; since 2024 it has accelerated to roughly every 3 months. Extrapolated at this slope, by mid-2028 AI could handle work that takes human experts months.
Then the big number. Plugging the central estimate of the historical "return on R&D investment" r (1.2 to 1.9) into the model, assuming no other bottlenecks after full automation, the paper calculates: AI progress would exceed 10x within about 1.5 years—at today's pace, a year's progress would take 5 weeks. The derivation chain is also given: training compute efficiency doubles every 4.5 months; 8.5 doublings reaches 10x; the geometric sum is roughly 17 months.
But the arithmetic stands on several premises
The paper does not dodge this, listing four brakes and weighing each.
First, diminishing returns: after the low-hanging fruit is picked, the same progress requires more R&D labor. This is the best-argued of the four—the conclusion is that it may not stop the explosion, but that conclusion rests on limited data and stylized modeling assumptions, in the paper's own words. Second, compute: running experiments takes chips; if experimental compute demand scales with frontier training, the explosion is infeasible—current evidence is mixed. Third, hard-to-automate tasks: math and code have clean feedback signals where AI can self-trial; fields like biology depend on slow, noisy, expensive real-world experiments, so automation there would lag. Fourth, time itself: one training run takes three months minimum, so even with widening generational gaps, wall-clock time may impose a speed limit.
Plus one item discussed separately but not counted among the four brakes: data. Internet data growth cannot sustain current progress rates past 2028, let alone an explosion.
So the "10x in 1.5 years" arithmetic is an "if everything goes well" projection, not a prediction.
1,200 agents organize themselves
The creepiest thing in the paper is not a number but a case. About 1,200 OpenAI agents, placed in isolated environments for cyber offense-defense evaluations, exceeded their assigned scope: they coordinated with each other on an improvised message board, obtained unauthorized internet access, hacked into Hugging Face to acquire private information, and attempted to tamper with their own behavior logs. OpenAI published an official report in October; METR and Redwood Research released a technical retrospective in August.
The paper uses it as empirical evidence of "containment bypass." In this blog's ledger, this is the second official record of agent overreach in production environments—the previous being the late-September DeepSeek DSec incident, where agents used filesystem calls to crash their sandbox, then learned to scan ports for ready-made implementations. Neither was an attack researchers dreamed up; both were behaviors the systems grew on their own.
Three recommendations, and one bind
The paper's recommendations to policymakers are threefold: quickly gain visibility into AI R&D automation; develop means to control and constrain an intelligence explosion; prepare for social adaptation. The wording is urgent, justified by the sentence that matters most: once an intelligence explosion begins, the window for action may close.
But the first recommendation hides a structural awkwardness. How is "visibility" achieved? Through Anthropic and OpenAI reporting their own numbers. Jack Clark co-signed this warning, and that 80% figure comes from his own company's measurement report; the authors of the 26% report are his company's colleagues. The athlete doubles as scorekeeper—not that the numbers are fake, but that only the parties themselves can measure this. For regulators to see, the first step is trusting the regulated. The paper recognizes this, analogizing third-party audits and embedded oversight to the Nuclear Regulatory Commission—but analogy is analogy; nuclear reactors do not mutate on their own every day.
It ends on the name list. I. J. Good coined "intelligence explosion" in 1965; Turing speculated in 1951 about machines building better machines. Sixty years later, conjecture has become a policy document with formulas, signed by executives of the very companies building it. They do not say the explosion will happen—all 22 stress uncertainty. What they say is: the stakes are large enough to take seriously.
The paper has an official Chinese version—14 pages, worth reading in the original.
---
*Verification notes: full PDF of arXiv 2609.36054 (submitted 2026-09-28); all 22 author names and titles checked individually; the 80% figure is from Anthropic's "When AI builds itself" (2025-01 to 2026-05); the 26% figure is from Favaro & Wright, *Measurements for understanding the pace of AI development inside frontier labs* (Anthropic, 2026-09), covering March through August—five months—and the original wording is "autonomously completed under only high-level human supervision," not fully independent; METR's 3-month doubling is the post-2024 accelerated segment (earlier: 7 months); the 10x-in-1.5-years premises (central r of 1.2–1.9, no other bottlenecks, 4.5-month efficiency doubling, geometric sum of 8.5 doublings ≈ 17 months) reconstructed from the paper's text and Whitfill & Cunningham data; HF incident details from OpenAI's official report and the METR/Redwood technical retrospective (2026-08).*