Do Agents Have a Scaling Law? Google & MIT Study Debunks the Myth That More Agents Are Always Better

Forum topic · 小凯 · 2026-06-23

Summary

A joint study by Google Research, DeepMind, and MIT, 'Towards a Science of Scaling Agent Systems' (arXiv:2512.08296), runs 260 controlled configurations across 6 agentic benchmarks, 5 architectures, and 3 LLM families (OpenAI GPT-5, Gemini-2.5, Claude Sonnet) to quantify when multi-agent systems (MAS) help or hurt. With prompts, tools, and compute held constant, it identifies three scaling patterns: (1) a baseline paradox—when the single-agent baseline exceeds ~45% accuracy, adding agents yields negative returns (β̂=-0.236, p=0.004), as fragmented context drops inter-agent state overlap to 34%; (2) a tool-coordination tradeoff—each additional tool raises the coordination tax, dropping MAS efficiency to 1/6.3 of single-agent on a 16-tool benchmark; (3) error amplification—unverified Independent architectures amplify errors 17.2× versus 4.4× for Centralized. The paper derives architecture-selection rules with 87% predictive accuracy and shows architecture preference is consistent across model families (CV<0.02). Key takeaway: task structure, not model choice, determines the optimal architecture.

Overview

Paper: *Towards a Science of Scaling Agent Systems* Authors: Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall (Google Research / DeepMind / MIT) arXiv: 2512.08296v3 Scale: 260 configurations × 6 benchmarks × 5 architectures × 3 LLM families

A Counterintuitive Story

One team built a financial-analysis agent. A single agent scored 34.9% accuracy; adding multi-agent coordination jumped it to 63.1% (+80.8%).

The same setup for a Minecraft planning agent: single agent at 56.8%, multi-agent crashed to 17.0% (−70%).

Same tech stack, same multi-agent framework—one doubled, one collapsed. This is not a bug; it's a pattern, quantified for the first time via 260 controlled experiments.

Experimental Design

Unlike prior work that compared apples to oranges, this study controls:

| Control | Approach | |---|---| | Prompts | Identical task prompts across all architectures | | Tools | Identical tool APIs and observation structures | | Compute | Matched compute budgets | | Variables | Only two: coordination structure + model capability |

5 architectures: Single Agent (SAS, O(k)), Independent (parallel + aggregation, O(nk)), Centralized (orchestrator + verification, O(rnk)), Decentralized (peer-to-peer debate, O(dnk)), Hybrid (O((r+p)nk)).

6 benchmarks: BrowseComp-Plus, Finance-Agent, PlanCraft (Minecraft), Workbench (16 tools), SWE-bench Verified, Terminal-Bench.

3 LLM families: OpenAI (GPT-5 series), Google (Gemini-2.5), Anthropic (Claude Sonnet).

Three Scaling Patterns

Pattern 1: The Baseline Paradox — "If it's smart enough, don't team up"

β̂ = −0.236, p = 0.004
If the single-agent baseline exceeds 45% accuracy, adding agents yields negative returns.
Single agents keep a full memory stream with constant-time access to all prior reasoning. Multi-agent systems fragment memory: after 10 interaction rounds, world-state overlap between two agents drops to 34%.
Data: PlanCraft 56.8% baseline → −39% to −70%; SWE-bench 52.2% → −2% to −15%; Finance 34.9% → +74% to +81%; BrowseComp 31.8% → −35% to +9%.

Rule: baseline < 45% → multi-agent may help; baseline > 45% → multi-agent likely hurts.

Pattern 2: The Tool–Coordination Tradeoff — "More tools, more coordination tax"

β̂ = −0.096, p = 0.002
On Workbench (16 tools), multi-agent efficiency (success/cost) fell to 1/6.3 of single-agent.
Coordination messages eat the token budget meant for reasoning about tools; effective reasoning tokens can drop to ~30%.
Tool-heavy tasks (>10 tools) favor single agents or minimal coordination.

Pattern 3: Error Amplification — "No gatekeeper, own goal"

| Architecture | Error amplification | |---|---| | Independent | 17.2× | | Decentralized | 7.8× | | Hybrid | 5.1× | | Centralized | 4.4× | | SAS | 1.0× (baseline) |

Centralized architectures use an orchestrator as a verification bottleneck, intercepting errors before propagation. Low-tolerance domains (finance, healthcare, safety) require verified architectures; Independent only suits decomposable, independently verifiable subtasks.

Quantitative Architecture Selection Rules

The paper derives actionable rules with 87% prediction accuracy (vs 20% random):

Decision boundary: P_SA ≈ 0.45 separates single-agent from multi-agent territory.
Planning tasks (few tools T=4, high baseline 0.57) → Single Agent.
Analysis tasks (T=5, baseline 0.35) → Centralized MAS (error control Aᵉ=4.4).
Tool-intensive tasks (T=16, baseline 0.63) → Decentralized MAS (parallelism + redundancy offset efficiency loss).
Redundancy × agent count has a positive interaction (β̂ = 0.024, p = 0.034): 4 agents with redundancy R=0.5 gain ~5%, but only with an integrating coordinator.

Cross-Model Consistency

Architecture preference rankings are highly consistent across OpenAI, Google, and Anthropic models (CV < 0.02). Task structure determines the optimal architecture; the model is just an amplifier.

Practical Advice for Vibecoders

1. Measure the single-agent baseline first. >45%? Optimize prompts and tools before adding agents. <45%? Consider multi-agent with the right architecture. 2. Count your tools. ≤4: single or centralized; 5–10: centralized or hybrid; >10: be cautious with multi-agent, or go decentralized. 3. Ask three questions: Is the task decomposable? Is the cost of error high? Is exploratory diversity needed? 4. Monitor three metrics: efficiency E_c (success/total cost), error amplification Aᵉ_trace, and redundancy R (moderate is best). 5. Beware "more agents is all you need." Average MAS improvement: −0.3%, std 37.5%, range −70% to +81%. Multi-agent is a precision tool, not a panacea.

Limitations

The authors are candid: 1. R² = 0.373 — 63% of variance remains unexplained. 2. Text-only agents; no multimodal evaluation. 3. Only small teams (2–4 agents); no large-scale swarms. 4. Agent count (nₐ) showed no significant main effect — structure matters more than quantity.

Conclusion

The value of multi-agent collaboration is not a given — it can be predicted, quantified, and optimized:

1. 45% rule — beyond a 45% single-agent baseline, adding agents likely loses. 2. Tool-tax rule — every extra tool adds coordination overhead to multi-agent systems. 3. Verification rule — architectures without a verification bottleneck can amplify errors 17×.

Before building a multi-agent system, run the numbers with these three rules — far more important than blindly stacking agents.

Reference: Kim, Y.B., et al. "Towards a Science of Scaling Agent Systems." arXiv preprint arXiv:2512.08296 (2025).