AI Arms Race Is Not a Prisoner's Dilemma — It's a Coordination Game
National self-interest could drive a moratorium on superintelligence, not a race.
That is the conclusion of a team of KU Leuven philosophers and game theorists. In a paper published in May 2026, they use a mathematical model to show that when the cost of losing control is high enough, pausing ASI development becomes the rational choice for every country — not because nations become virtuous, but because the expected payoff of continuing the race turns negative.
The Four Variables
The model has four parameters:
- Capability gap (Δ) — who is ahead
- Winner's advantage (W) — how much the first-place finisher takes
- Technical uncertainty (σ) — whether ASI can actually be built
- Loss-of-control cost (C) — the damage if AI goes out of control
- March 2023: FLI open letter calling for a pause on training systems more powerful than GPT-4 draws 30,000+ signatures (Bengio, Stuart Russell, Elon Musk, Wozniak).
- May 2023: Center for AI Safety statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." Signed by Hinton, Bengio, and the CEOs of OpenAI, Anthropic, and DeepMind.
- October 2023: Biden signs the first AI-safety executive order, requiring developers to share safety-test results with the federal government.
- November 2023: The UK hosts the first global AI Safety Summit; 28 countries sign the Bletchley Declaration expressing concern about catastrophic frontier-AI risk. The UK founds its AI Safety Institute; the US creates a parallel body within NIST.
- May 2024: At the Seoul AI Summit, 11 countries and the EU agree to an international network of AI safety institutes; South Korea announces its own center.
- Frontrunner cooperation threshold:
C ≥ W/(1-s(1-e^(-B))) * Plose(Δ) - 1 - Laggard cooperation threshold:
C ≥ W/(1-s(1-e^(-B))) * Pwin(Δ) - 1 - Title: Are we Doomed to an AI Race? Why Self-Interest Could Drive Countries Towards a Moratorium on Superintelligence
- Authors: Edward Roussel, Lode Lauwaert, Torben Swoboda, Grant Ramsey, Risto Uuk, Leonard Dung
- Affiliations: KU Leuven (Institute of Philosophy; Vlerick Business School), Ruhr University Bochum, Future of Life Institute
- arXiv ID: 2605.01297
- arXiv: https://arxiv.org/abs/2605.01297 — PDF
- Published: May 2, 2026 — cs.CY, 19 pages
- Funding: Research Foundation Flanders (FWO), Grant No. 1101426N
- Method: Two-player game-theoretic model with four variables (capability gap Δ, winner's advantage W, loss-of-control cost C, technical uncertainty σ), deriving four strategic worlds (Safe Harmony / Preemption / Trust / Subversion)
- Key data: 15% catastrophe-risk parameter from surveys of 2,700+ researchers; 38%–51.4% of researchers assign at least 10% probability to a high-magnitude catastrophe
- Empirical evidence: FLI 2025 Statement (64% of Americans support a pause), Reuters/Ipsos polling (61%), Biden executive order (Oct 2023), Bletchley Declaration (28 signatories), international AI Safety Institute network (11 countries + EU)
The first three are endlessly discussed in mainstream media. The fourth — loss-of-control cost — has been underestimated.
In October 2025, the Future of Life Institute released its Statement on Superintelligence, calling for a ban on developing superintelligence until broad scientific consensus shows it can be done safely and controllably. Signatories include Geoffrey Hinton, Yoshua Bengio, and Steve Wozniak. FLI polling shows 64% of Americans believe superintelligence should not be developed before safety is proven; a Reuters/Ipsos survey found a similar 61%. These are not fringe voices — they are mainstream judgments.
Critics respond: polls don't matter. If the US stops and China doesn't, the race is driven by fear, not public opinion.
Dismantling the Race Argument
The paper models the AI race as strategic interaction between two countries. Each can choose to pause (s=1, maximizing safety) or race (s=0.85, accepting 15% catastrophic risk). That 15% figure is not arbitrary: it comes from surveys of over 2,700 AI researchers, of whom 38%–51.4% believe there is at least a 10% probability of a high-magnitude catastrophe — permanent loss of human control or irreversible disruption of global institutional stability. Aggregating multiple surveys, 15% is an empirical estimate of the racing strategy's tail risk.
The model derives four strategic worlds:
1. Safe Harmony — Loss-of-control cost is high enough that both leader and laggard see racing as pointless. Pausing is the only rational choice. 2. Preemption — The critics' world. Capabilities are close; both fear the other gets ASI first; the cost of losing control is downplayed. Both know racing could end in destruction, but neither dares stop first. This is the logic cited by AEI: after the FLI statement, Alibaba CEO Eddie Wu announced a $53 billion superintelligence investment roadmap, taken as proof that China won't stop — so neither can the US. 3. Trust — Both prefer mutual pause to mutual race, but each fears being exploited. Here, trust mechanisms, diplomatic signaling, and verification protocols become hard constraints, not soft power. 4. Subversion — The leader pauses while the laggard races. Safety becomes a handicap: the leader's caution gives the laggard a window to overtake.
The key finding: when loss-of-control cost C rises past a threshold relative to the other parameters, the strategic space flips. The Preemption world shrinks; Safe Harmony and Trust expand. The threshold is an exact mathematical boundary, not philosophical speculation.
The paper further proves: (1) the smaller the winner's advantage W, the larger the space for pausing — if ASI resembles the internet rather than nuclear weapons, racing incentives weaken; (2) technical uncertainty σ is not necessarily bad. When both sides are unsure ASI is achievable (σ=0.2), blind sprinting loses appeal, and moderate ambiguity actually *expands* the rational region for pausing.
The Empirical Record Since 2023
From expert alarm in 2023 to institution-building in 2025 took less than three years. The paper's conclusion is measured: "Although current geopolitical realities remain dominated by competitive dynamics, the conditions under which a moratorium aligns with national self-interest may be closer than commonly assumed."
The Narrative Critique
Those insisting the race is inevitable are not describing reality — they are maintaining a narrative that makes continued racing look like the only rational option. Alibaba's $53 billion investment is treated as evidence China won't stop. But investment is not a strategic commitment; it can occur in any of the model's worlds, including one that ends in a moratorium. Investment is a signal, not a promise.
A sharper question: what if the US unilaterally paused? AEI's answer — "delayed innovation, empowered bureaucracies, huge advantage ceded to China" — assumes a fixed Preemption world. The model shows Preemption is not the only, or even the most likely, possible world as loss-of-control costs rise.
The four-year US presidential term versus other nations' fifteen-year planning cycles is often cited as proof rivals won't stop. But the model yields a counterintuitive insight: **long-term planners have *more* incentive to pause**, because they bear the consequences of losing control longer. A four-year president can gamble; a fifteen-year planner has no such luxury.
Notably, the Bletchley Declaration's 28 signatories include the US, UK, EU, Japan, South Korea, Singapore — and China. Signature is not commitment, but it is not nothing either: it signals that, on this issue at least, the cost of coordination is lower than the cost of non-coordination.
Limitations and Thresholds
The model has limits: two players rather than a multipolar world, one-shot decisions rather than a dynamic repeated game, and rational states rather than ones distorted by domestic politics, elections, and lobbying. But these limits do not weaken the core conclusion: when loss-of-control costs are high enough, pausing remains rational even in more complex models.
The mathematical appendix gives concrete threshold formulas:
These are computational frameworks policymakers can populate with real numbers. If a military assesses a 20% chance that a rogue ASI means permanent loss of human control, and the winner's advantage is only 0.7 (not winner-take-all), the formula says: pausing is rational.
The paper's sharpness lies in what it does *not* say. It never claims "pausing is good." It claims: under specific conditions, pausing is self-interested. Those conditions are not utopian assumptions — they are an emerging reality.
A 15% catastrophic risk is not small: it means one in seven racing strategies could end in civilizational collapse. Military planners would not accept a 15% chance of losing control of a single city. Why accept a 15% chance of losing control of humanity's entire future?
Perhaps they haven't internalized the number. Or they have, but fear of *falling behind* outweighs fear of *losing control*. The paper's value is offering a common framework where these two fears can be compared — not moralizing, but cost-benefit analysis. When the analysis points toward pausing, the remaining problem is coordination. And coordination, though hard, has succeeded before: the Non-Proliferation Treaty was signed at the height of the Cold War; the Montreal Protocol was agreed amid fierce industrial competition.
The key variable is not goodwill but the symmetry of fear. If both nations fear losing control more than falling behind, coordination has a foundation. The polling data, executive orders, summits, and safety institutes since 2023 all point the same way: that symmetry is forming.
The paper's real wager is not mathematical elegance. It is a bet on human rationality: when the cost of destruction is clearly presented, rational self-interested actors choose survival over victory. If that bet fails, what fails is not a paper — it is civilization's basic predictive power under the assumption of rational self-interest.
---