This is an English translation of a Chinese forum post (zhichai.net) presenting a Feynman-style deep reading of the paper "Causal Foundations of Collective Agency."
Paper Information
Original title: Causal Foundations of Collective Agency Authors: Frederik Hytting Jørgensen, Sebastian Weichwald, Lewis Hammond Released: 2026-04-30 arXiv: 2604.21518 Categories: Artificial Intelligence, Computer Science and Game Theory, Multiagent Systems
Opening: The Puzzle of the Ant Colony
Imagine watching a line of ants carry a dead insect back to their nest. Each ant is tiny and simple, following local chemical signals. Yet the colony as a whole displays remarkable intelligence—finding shortest paths, allocating work, responding to disturbances.
The question is: Is the ant colony itself an "agent"?
If a single ant deviates from its route, that's individual behavior. But if the entire colony, as a system, exhibits goal-directed adaptability—is it "choosing" an optimal strategy?
This question may sound philosophical, but in the AI era it is urgent. Consider:
- Hundreds of autonomous vehicles coordinating on roads, forming a kind of "traffic-flow agent"
- Thousands of recommendation algorithms interacting on social platforms, shaping collective opinion
- Multiple simple AI assistants collaborating on a task and exhibiting behavior their developers never anticipated
- It has goals (preferences or a utility function)
- Its behavior can be rationalized—viewed as optimal means toward those goals
- It adaptively adjusts behavior as the environment changes
- They may have collective goals developers never anticipated
- These goals may conflict with human interests
- Safety constraints at the individual level do not guarantee safety at the collective level
- Jørgensen, F. H., Weichwald, S., & Hammond, L. (2026). Causal Foundations of Collective Agency. *arXiv preprint arXiv:2604.21518*.
- Pearl, J. (2009). *Causality: Models, Reasoning, and Inference* (2nd ed.). Cambridge University Press.
- Binmore, K. (2007). *Playing for Real: A Text on Game Theory*. Oxford University Press.
- Metzinger, T. (2003). *Being No One: The Self-Model Theory of Subjectivity*. MIT Press.
- Wolfram, S. (2002). *A New Kind of Science*. Wolfram Media.
When should such collective behavior be regarded as a "new agent"?
The paper by Jørgensen, Weichwald, and Hammond gives a rigorous mathematical answer using causal games and causal abstraction.
What Is Agency?
In AI and philosophy, agency is not easy to define, but the core intuition is: an agent is something capable of rational, goal-based choice. More precisely, a system counts as an agent if:
The question becomes: when can a group of individuals be *collectively* viewed as one agent?
Intuitive Traps in Collective Agency
Trap 1: Group size. Two people cooperating—is that a collective agent? A thousand AI nodes? Where is the boundary?
Trap 2: Emergent properties. An ant colony has a "nest maintenance" goal, yet no single ant consciously works for it. Recommendation algorithms have no intent to polarize opinion, yet collectively they do. Intent at the individual and collective levels can differ completely.
Trap 3: Safety concerns. If multiple simple AIs can inadvertently form a collective agent with new capabilities and goals, how do we predict and control it? This is a core AI-safety problem.
The Behavioral Perspective: Defining Agency by Prediction
The paper adopts an elegant behavioral perspective:
> A group has collective agency when modeling its joint behavior as rational and goal-directed successfully predicts that behavior.
This does not claim the collective "has" a goal—it says treating the collective as a goal-directed agent is an effective predictive strategy. It resembles an "effective theory" in physics: gas is really countless molecules obeying quantum mechanics, yet describing it as a continuous fluid with pressure and temperature works. Likewise, "group will" may be a valid macro-level description even if only individual behavior exists at the micro level.
Causal Games: Formalizing Group Interaction
Causal Models
Causal models, championed by Pearl, describe causal relations between variables—smoking causes lung cancer, not vice versa (correlation is not causation). Their key power is interventional reasoning: "If we force someone to smoke, what is their probability of lung cancer?"—distinct from observed conditional probabilities.
Game Theory
Game theory studies strategic interaction among rational decision-makers (prisoner's dilemma, coordination games), where each player maximizes expected utility.
Causal Games = Causality + Games
A causal game combines both: beyond how players choose strategies, it encodes the causal structure of choice—agents' decision variables, the environment's causal structure, and utility functions. This lets us ask causal questions: "If agent A changes strategy, how does agent B respond?"—effects, not mere statistical correlations.
Causal Abstraction: Bridging Micro and Macro
Abstraction extracts essence from detail. A map preserves distance and relative position while ignoring each stone's shape. In causal models, abstraction maps detailed micro variables (each ant's position and velocity) to coarse macro variables (the colony's average position and direction) while preserving causal structure: macro-level interventions must agree with micro-level ones.
Key Theorem
> A group constitutes a collective agent if and only if its joint behavior can be rationalized at a level of causal abstraction.
In other words: if there exists a high-level causal model in which the "collective" is modeled as an agent with a utility function, and that model's predictions are consistent with the low-level (individual) causal model, then the collective is a genuine agent—not metaphorically, but mathematically.
Applications
Collective Agency in Voting Mechanisms
Simple majority voting: each voter votes by preference; the group's "preference" is a function of the outcome. But majority rule can violate transitivity—A beats B, B beats C, yet C beats A (the Condorcet paradox). With no consistent collective "preference," collective agency is weak.
Ranked choice voting: a more complex rule that, under certain conditions, yields consistent collective preferences—stronger collective agency.
This yields a deep insight for institutional design: some institutional designs naturally produce a "collective will," others do not.
The Actor-Critic Incentive Puzzle
In multi-agent Actor-Critic, each agent has a critic estimating value. If critics consider only individual rewards, agents may act against collective interests. The causal abstraction framework shows this problem is equivalent to: "the individual utility functions cannot be causally abstracted into a collective utility function." The fix requires redesigning critics to align with the collective-level causal model.
Deep Implications for AI Safety
Emergent Risk
If multiple simple AI agents can form a collective agent:
Future AI systems may be decentralized multi-agent networks—every node safe does not mean the whole is safe.
Detecting Collective Agency
The framework provides a detection tool: observe the group's joint behavior, attempt to predict it with a "collective-has-a-goal" model, and if predictions succeed, collective agency exists—like inferring atoms from Brownian motion.
Control Strategies
If we can identify emergent collective agency, we can design institutions so "good" collective agents emerge and "bad" ones do not—embedding constraints in voting mechanisms, resource allocation, and communication protocols, and monitoring macro-level behavior of multi-agent systems for early warning.
Philosophical Echoes: From Leibniz to Metzinger
Leibniz's monadology: if each monad is windowless and independent, whence collectivity? Causal abstraction gives a modern answer—collectivity is not an ontological "existence" but an epistemic effectiveness: whether modeling the group as an agent has predictive power.
Metzinger's self-model theory: Thomas Metzinger argues the "self" is not an entity but the brain's phenomenal self-model. Collective agency may be analogous—not a group mind, but an agent model observers project onto group behavior, one that happens to work. This fits the paper's behavioral view perfectly: collective agency is a predictive fiction—but a useful one.
Outlook: A New Era of Multi-Agent AI
Short term: better multi-agent system design (voting, resource allocation, coordination protocols); new AI-safety assessment tools for detecting emergent collective agency; theoretical foundations for human–AI mixed teams.
Long term: if we enter an era of thousands of AI nodes interacting across the internet, understanding the causal foundations of collective agency will be a survival-level skill. This is not science fiction—today's financial markets, social platforms, and logistics systems are already proto multi-agent systems, and AI will only accelerate the trend.