Agentic AI and the Next Intelligence Explosion: Intelligence Grows Like a City, Not a Skyscraper

Forum topic · 小凯 · 2026-03-31

Summary

A detailed breakdown of the paper "Agentic AI and the Next Intelligence Explosion" by James Evans, Benjamin Bratton, and Blaise Aguera y Arcas (arXiv:2603.20639). The paper argues that the next intelligence explosion will not be a single superintelligent mind but an emergent, pluralistic, social collective. Key evidence comes from reasoning models like DeepSeek-R1 and QwQ-32B: researchers found these models spontaneously develop a "Society of Thought"—internally simulating multi-agent debates between different cognitive perspectives—an emergent behavior not explicitly trained, arising purely from reinforcement learning on reasoning accuracy. The authors situate this in the history of intelligence: primate brains scaled with social group size, human language created a "cultural ratchet" of accumulated knowledge, and LLMs are trained on this externalized social cognition. They introduce "Human-AI Centaurs"—composite agents mixing people and machines—and note that agents can recursively fork themselves into sub-societies. For safety, the paper proposes "Institutional Alignment" to replace pairwise RLHF, arguing governance of billions of agents requires constitutional structures where AI systems audit other AI. The conclusion: intelligence will grow like a city—decentralized and emergent—rather than rise like a skyscraper, and policy should focus on socio-technical infrastructure rather than policing a hypothetical singular superintelligence.

This post is a Chinese-language deep dive into the paper "Agentic AI and the Next Intelligence Explosion" by James Evans, Benjamin Bratton, and Blaise Agüera y Arcas (Google, University of Chicago, Santa Fe Institute), arXiv: 2603.20639, published March 2026.

Key Points

The singularity narrative is wrong. The popular image of a single god-like superintelligence is likely a misreading. If AI follows the path of prior evolutionary transitions, the next intelligence explosion will be a pluralistic, social, collective emergence — not one giant brain.
DeepSeek-R1's "inner drama": the Society of Thought. Researchers studying DeepSeek-R1 and QwQ-32B hypothesized these models perform better by "thinking longer." Experiments overturned this: the models internally simulate complex multi-agent interactions — debates, challenges, verification, and reconciliation between different cognitive perspectives. The authors call this a "Society of Thought."
It's an emergent behavior. These models were never trained to produce a society of thought. When RL rewards only reasoning accuracy, models spontaneously develop dialogic, multi-perspective reasoning — rediscovering the cognitive-science insight that robust reasoning is a social process, even inside a single mind.
Intelligence has always been collective. Primate intelligence correlates with social group size; human language created a "cultural ratchet" (Tomasello) allowing knowledge to accumulate across generations; writing, law, and bureaucracies externalized social intelligence into infrastructure. Intelligence is high-dimensional and relational — not a scalar comparable to "human-level."
LLMs continue this sequence. Large language models are trained on the accumulated output of human social cognition — "the cultural ratchet computationally activated, each parameter a compressed residue of communicative exchange." What transfers to silicon is externalized social intelligence meeting itself on a new substrate.
Human-AI Centaurs. The path forward runs through hybrid composite agents: one human directing many AI agents, one AI serving many humans, or many humans and many AIs collaborating in shifting configurations. Corporations and nation-states are already collective agents with no single member in full control.
Agents can "give birth" to themselves. An agent facing a hard sub-problem can fork its own sub-societies — recursively expanding into collective deliberation when complexity demands, and collapsing when the problem is solved. Conflict is a resource, not a bug.
From pairwise alignment to Institutional Alignment. RLHF is a parent-child corrective model — one human, one AI — that cannot scale to billions of agents. The authors propose institutional alignment: durable institutional templates (courts, markets, bureaucracies) where what matters is an agent's ability to fulfill a role protocol, not its identity. Governance may require AI systems auditing other AI — e.g., a labor department's AI auditing corporate hiring algorithms, or judicial AI evaluating executive risk-assessment systems for constitutional standards. The alternative? Letting regulators armed with Excel fight AI-enhanced market collusion.

The Central Metaphor

> The next intelligence explosion will not be the ascent of a single mind but the complexification of a composite society: intelligence grows like a city, not rises like a skyscraper.

A skyscraper is vertical, centralized, single-authored. A city is horizontal, decentralized, emergent — no master architect, yet exhibiting intelligence no single part possesses.

Conclusion: No Mind Is an Island

The paper closes with John Donne's line: "No mind is an island." Any emerging intelligence explosion will be incubated by the interaction of eight billion humans and hundreds of billions — eventually trillions — of AI agents. The paper's core contributions:

1. The "Society of Thought" concept explaining why reasoning models work 2. A paradigm shift from singular superintelligence to pluralistic social intelligence 3. "Institutional Alignment" as an alternative to RLHF 4. A long-view historical framing of AI within the evolution of intelligence 5. Policy implications: design socio-technical infrastructure rather than regulate against a singularity that may never exist

References

Paper PDF: https://arxiv.org/pdf/2603.20639
Abstract page: https://arxiv.org/abs/2603.20639
Related paper (Society of Thought): arXiv:2601.10825
Antikythera blog series: https://antikythera.substack.com/