The Trouble With Collective Intelligence: 260 Experiments Reveal the Harsh Truth About Multi-Agent Collaboration

Imagine having nine brilliant people on your team, each with Stanford-level intelligence. You ask them to solve a problem together. The result? Nine working together underperform one working alone.

That's not just a dark joke from management science. It happened to AI, and far more brutally than you might expect.

Researchers from Google Research and MIT ran 260 rounds of experiments to verify something everyone assumed was common sense: can multiple AI agents collaborating outperform a single one? The answer made even them gasp — on average, the collaboration benefit of multi-agent systems is close to zero.

Yes, you read that right. Nine models together lose to one model working alone. On some tasks, multi-agent collaboration slashed performance by 70%; on others, it produced gains of 80% out of nowhere.

A strange asymmetry. And behind it lies a more unsettling truth: we keep adding agents to systems, but almost nobody stops to ask — when should you add them, and when should you stop?

The paper *Towards a Science of Scaling Agent Systems* (Kim et al., 2025) put this question on the operating table for the first time. Their approach was hardcore: no flashy new frameworks. Controlled experiments that decompose "collaboration" into quantifiable variables, fed into a regression model, to see what pushes performance up and what drags it down.

Let's dissect this surgery.

A Taxonomy of the Agent World

Before discussing "collaboration," one thing must be settled — who is collaborating with whom.

The paper splits agent systems into two paths: Single-Agent Systems (SAS) and Multi-Agent Systems (MAS). A single agent talking to itself — self-reflection, chain-of-thought reasoning — does not count as multi-agent collaboration. The bar is strict: multiple LLM instances must communicate via structured message passing, shared memory, or orchestration protocols.

That yields five architectural blueprints:

| Architecture | One-line summary | Communication pattern | Key cost | |------|-----------|---------|--------| | Single-Agent | One person carries everything | None | Zero overhead, but single-path exploration | | Independent | Everyone works separately, results merged | Agent→aggregator (one-way) | 58% extra overhead, no cross-validation | | Centralized | A "manager" coordinates everything | Orchestrator↔agents (star) | 285% overhead, but a validation bottleneck | | Decentralized | All-to-all discussion reaches consensus | Full-to-full | 263% overhead, no central control | | Hybrid | Manager plus peer-to-peer exchange | Star + peer edges | 515% overhead, the most expensive |

> Tip: "Communication overhead" (O%) means the extra total tokens a multi-agent system burns relative to a single agent. Hybrid's 515% means it consumes roughly six times the compute.

These five architectures resemble five survival strategies in the animal kingdom. The question: in which environments does each strategy survive?

The Experiment: 260 Rounds of a "Hunger Games"

The experimental design is a textbook of controlled variables:

First, lock every lockable variable. Same tool APIs, same prompt templates, same compute budget allocation. A team of n agents gets proportionally fewer reasoning rounds each — closing the loophole of "it scored higher because it burned more compute."

Second, cover enough ecological niches. Six benchmarks spanning financial analysis to Minecraft planning, software engineering to command-line operations:

| Benchmark | Task type | Tools | In one line | |---------|---------|--------|--------| | Finance-Agent | Financial reasoning, decomposable | 5 | Junior analyst: read news, parse filings, synthesize | | BrowseComp-Plus | Web information retrieval | — | Locate information across sites | | WorkBench | Business activity planning | 16 | Most tools, a "Swiss army knife test" | | SWE-bench Verified | Software engineering | 7 | Fix GitHub issues, programmer's daily grind | | Terminal-Bench | CLI operations | 2 | Sysadmin, security, ML tasks | | PlanCraft | Sequential planning | 4 | Survival guide in Minecraft |

Third, three LLM families, nine models. OpenAI (GPT-5-nano/mini/5), Google (Gemini-2.0/2.5 Flash, 2.5 Pro), Anthropic (Claude Sonnet 3.7/4/4.5), spanning an Intelligence Index from 42 to 71.

Each benchmark × three models per family × five architectures = 260 independent configurations, each run on 50–100 instances. This isn't parameter tuning — it's a system-level controlled experiment.

Three Scaling Laws

All data went into a 20-parameter regression model. Once the noise settled, three laws emerged.

Law One: The Baseline Paradox — Stronger Single Agents, Worse Collaboration

The paper's core finding. An interaction term has a strikingly large coefficient: P_SA × log(1+n_a) has β = -0.236, p = 0.004.

In plain language: once single-agent baseline accuracy crosses roughly 45%, every additional agent starts dragging performance down. Not diminishing returns — outright loss. The paper calls it the "capability-saturation effect."

PlanCraft is the starkest example. Single-agent average accuracy is 56.8% — "good enough." Multi-agent results: Independent crashes to 17% (-70%); even the best Hybrid only reaches 34.6% (-39.1%). Why? Execution traces make it obvious. A single agent does PlanCraft in three clean steps: look up recipe → place materials → craft. Meanwhile a Centralized MAS has Agent 1 researching the recipe (redundant), Agent 2 checking inventory (also redundant), and only Agent 3 actually executing. Three smart people spending twice the time on something one person could do in three seconds.

> Tip: This is the "coordination tax" — the fixed cost of communication, synchronization, and consensus that every multi-agent architecture pays. When the task is simple enough, the tax becomes a net loss.

Law Two: More Tools, More Expensive Collaboration

The second law completes the sentence: when tools multiply, don't add agents either.

In the regression, E_c × T has β = -0.096, p = 0.002. T is tool count, E_c is coordination efficiency. The interaction is negative — the more tools, the harder coordination becomes for multi-agent architectures.

WorkBench (16 tools) is the best illustration. The best MAS there (Decentralized) beats a single agent by only 5.6%; Independent loses 11%. Sixteen tools mean a huge action space per agent. You touched Slack, I changed Google Calendar, and nobody knows what the third agent did. The cost of information fragmentation compounds exponentially in tool-dense environments.

Conversely, Finance-Agent has only five tools, yet Centralized delivered +80.8%. Five tools are just enough to split into focused subtasks, each assigned to one agent, with the orchestrator synthesizing at the end. That's the golden intersection of "task decomposability" and "tool count."

Law Three: The Geometry of Error Propagation

The paper introduces a subtle metric: the error amplification factor, defined as A_e^task = (1 - P_MAS) / (1 - P_SAS). Values above 1 mean the multi-agent system makes more errors than a single agent.

| Architecture | Error amplification | In one line | |------|------------|--------| | Single-Agent | 1.0× | Baseline | | Centralized | 4.4× | Validation bottleneck keeps errors contained | | Hybrid | 5.1× | Horizontal channels add contamination | | Decentralized | 7.8× | Everyone talks, errors echo | | Independent | 17.2× | No one checks anyone; catastrophic amplification |

Independent is the most dangerous — errors amplified 17.2-fold. It has neither centralized validation (like Centralized) nor peer debate (like Decentralized). Each agent works in isolation and hands output to the aggregator. Agent A makes a small error; nobody notices. It's a factory without quality control — defect rates can't fall, only rise.

The paper also classifies error types. Under Centralized, context omission dropped 66.8% and logical contradictions dropped 36.4%. But under Hybrid, numerical drift actually worsened — horizontal communication introduces more "second-hand information," and distortion grows with every hop.

The Anatomy of Errors

| Error type | Single-agent baseline | After Centralized | Change | |---------|---------------|---------------|--------| | Logical contradictions | 12.3-18.7% | 9.1% | ↓36.4% | | Numerical drift | 20.9-24.1% | 18.3% | ↓24% | | Context omission | 15.8-25.2% | 8.3% | ↓66.8% | | Coordination failure | — (MAS-only) | 1.8% | Newly introduced |

Centralized's error-reduction effect is striking — context omission cut by nearly two-thirds. The mechanism is simple: the orchestrator acts as gatekeeper, re-reviewing all agent output each round and calling out gaps.

But one detail is alarming: Hybrid's coordination failure rate hits 12.4%. Once horizontal channels open, the orchestrator loses control. Information flows through two channels simultaneously — the "official channel" and "side chats" — and maintaining consistency doesn't grow linearly. It explodes.

> Tip: "Coordination failure" means two or more agents give contradictory judgments about the same fact and the system fails to resolve the conflict. It's most common in Hybrid — because agents can communicate directly, they may form "clique consensus" that bypasses the orchestrator's global view.

A Navigation Map for Architecture Choice

The paper's trained regression model predicts the best architecture on held-out configurations with 87% accuracy. Compare: random choice gets 20%; picking by model capability alone gets 54%. Architecture-task alignment has real, quantifiable structure.

Condensed into a practical guide:

| If your task... | Recommended architecture | Typical scenario | |---------------|---------|---------| | Already done well by a single agent (>45%) | Single-Agent | PlanCraft, routine code generation | | Decomposable, few tools (≤5), single agent struggles (<35%) | Centralized | Financial analysis, research reports | | Many tools but single agent is OK (~60%) | Decentralized | WorkBench, multi-tool orchestration | | Extremely complex, needs deep verification | Hybrid (use cautiously) | Critical decisions with layered review |

The key decision variable is **P*_SA ≈ 0.45. Running multi-agent experiments without first measuring your single-agent baseline is like slamming into a wall with your eyes closed.

One often-missed detail: LLM families differ in architectural sensitivity. Anthropic models showed a unique "heterogeneous mixing benefit" in Centralized setups — mixing differently capable Claude models worked better than uniform teams. OpenAI models were more sensitive to communication quality; in Decentralized setups, poorly written messages hurt them faster than Google models. The maximum cross-family difference in architecture scaling slope was only 0.023 (CV < 0.02), so the overall trend is robust — but family "personality" matters.

The Efficiency Bill

| Architecture | Successes per 1,000 tokens | Relative efficiency vs single agent | |------|---------------------|-----------------| | Single-Agent | 67.7 | 1.00× | | Independent | 42.4 | 0.63× | | Decentralized | 23.9 | 0.35× | | Centralized | 21.5 | 0.32× | | Hybrid | 13.6 | 0.20× |

Hybrid burns six times the tokens of a single agent yet produces one-fifth the successes per thousand tokens. To match a single agent's total successes, Hybrid needs roughly 30x the token spend. "Inefficient" doesn't cover it — it's a money-burning race.

The paper also estimated dollar costs. OpenAI's Hybrid costs about $0.008 per 1% accuracy gain; switching to Anthropic models, it jumps to $0.024 — three times. The real cost of multi-agent systems isn't development; it's inference.

The Road Ahead: This Is Only the Beginning

The word "Towards" in the title is easy to miss. The paper admits this is a first step.

Notable limitations:

• R² is stuck at 0.37–0.41. More than half the performance variance remains unexplained — possibly subtle task structure, prompt wording, or agent-persona effects still outside the model's blind spots.
• Intelligence Index only spans 42–71. If far stronger models arrive — breaking 100, 200 — will these scaling laws hold? Cluster-robust inference says capability saturation is the most stable finding, but the definition of "saturation" may need rewriting.
• The largest team is only 9 agents. The paper itself finds reasoning rounds grow as a power law: T = 2.72 × (n + 0.5)^1.724. Extrapolating, a 100-agent team would need over 7,000 reasoning rounds before counting tokens or coordination overhead. The system might collapse economically before reaching that scale.
• Cluster-robust inference exposed hidden instability. After dataset-level clustering corrections, some predictors' standard errors inflated 2.9x. Cross-task generalization remains open — a law true on six benchmarks may wobble on the seventh.

1. Kim, Y., Gu, K., Park, C., et al.** *Towards a Science of Scaling Agent Systems.* arXiv:2512.08296v3, 2026. — The core paper: a quantitative framework for agent-system scaling across 260 configurations, 5 architectures, 6 benchmarks, and 3 LLM families. 2. Kaplan, J., McCandlish, S., Henighan, T., et al. *Scaling Laws for Neural Language Models.* arXiv:2001.08361, 2020. — The founding work on neural scaling laws and the dual reference point for "collaboration scaling." 3. Park, J. S., O'Brien, J. C., Cai, C. J., et al. *Generative Agents: Interactive Simulacra of Human Behavior.* UIST, 2023. — Foundational work on multi-agent interaction simulation. 4. Li, G., Hammoud, H., Itani, H., et al. *CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society.* NeurIPS, 2023. — A representative multi-LLM communication framework. 5. Chowdhery, A., Narang, S., Devlin, J., et al. *PaLM: Scaling Language Modeling with Pathways.* JMLR, 2023. — Systematic study of large-scale LLM training and scaling.