Why Small Parallel Agent Teams Beat Both Independent Agents and One Big Team
This post is a translated and edited version of a popular Chinese tech forum deep-dive on the paper *Test-Time Compute Scales Better with Parallel Agent Teams* (Wang, Chen, Li, Chalamala & Liu, arXiv:2611.09798), by researchers from Duke University, Modal Labs, Princeton, and NYU (Zhuang Liu's team).
The core idea
The author opens with a lottery analogy. Given 100 tickets' worth of budget:
- 100 independent tickets (Best-of-N sampling): many chances, but each is weak — returns saturate fast.
- One big pooled ticket (one large collaborative team): stronger per draw, but only one draw — no variance to exploit.
- Ten groups of ten, each pooling a ticket: the sweet spot — many strong draws.
- 12 hard, open-ended tasks across 6 benchmarks: KernelBench (GPU kernel optimization), MiniF2F (formal math proofs in Lean), NYU CTF Bench (binary exploitation), SWE-Bench Pro (real-world bug fixing), Taubench (travel booking), PINTO (geophysical inversion).
- Main model: Qwen3.8-27B, each agent runs 12 hours, communicating via a simple asynchronous forum.
- Teams are flat: no roles, no manager, no task allocator.
- Diminishing returns are halved in teams: going from 10 to 20 agents adds 1.27% per agent for teams vs. 0.64% for independents.
- Small-model teams vs. frontier models: at equal dollar cost, 27B-model teams beat GPT-5.6-sol independent sampling on 4 of 8 comparable tasks — collaboration structure can partially substitute for model scale.
- Emergent collaboration: agents spontaneously invented
[win]/[fail]tagging (adopted 88% of the time), propagated new best solutions within 10 minutes, showed social politeness, and shared artifacts like memory-optimization strategies in KernelBench. - Managers are harmful: teams with an explicit manager agent underperformed both flat teams and independent agents at every budget. Likely causes: single-point bottleneck, task lock-in, and coordination overhead consuming exploration budget.
- More tools ≠ better: adding tools (browser, executor, file system) did not consistently help and sometimes hurt — choice itself is a cognitive burden.
- For system builders: split your inference budget into 5–15 agent flat teams run in parallel, take the best result. No training required.
- For model buyers: a mid-size model with a good collaboration structure can rival larger models on hard tasks.
- For multi-agent researchers: don't over-engineer protocols — simple communication infrastructures let useful behaviors emerge; central coordination hurts on open-ended exploration tasks (though it may still help on procedural ones).
The paper finds that with a fixed budget of 100 agents, 10 teams of 10 beat 100 independent agents on 10 of 12 tasks, and beat 1 team of 100 on all 4 tested tasks.
The math
The expected best score across N samples is:
where \(a_N \approx \sqrt{2 \ln N}\) grows only logarithmically. This explains why Best-of-N saturates: the paper shows 50 independent agents already capture 81–100% of the gains from 100. Meanwhile, σ (score spread) is amplified linearly. Small teams raise μ through collaboration while keeping N > 1 to harvest variance. The authors compute that mean gains of only \(\sigma \cdot \ln(c) / \sqrt{2 \ln B}\) are needed to offset a c-fold cost increase from coordination — far below the measured 27%+ mean gain of 10-person teams over independent agents. A corollary: if collaboration widens σ from 1 to 1.3, just 8 teams match 100 independent agents on E[max].
Experiments
Key results
| Comparison | Result | |---|---| | 10×10 teams vs. 100 independent agents | teams win 10/12 tasks | | 10×10 teams vs. 1×100 team | teams win 4/4 tasks (by 8–30%) | | 10-agent team vs. 10 independent agents | team wins by ~10% (grows to ~27% at 100-agent budget) |
Limitations
1. The statistical model assumes near-normal score distributions; binary outcomes only fit qualitatively. 2. Only 10–11 team runs per task — limited statistical power. 3. Large-team forum indexing may have limited information flow (big teams might be even worse than reported). 4. Only one harness (async forum) tested; other topologies may shift the optimal team size. 5. Selecting the best team without a ground-truth verifier remains an open problem. 6. Cost models ignore hidden coordination infrastructure costs.
Practical takeaways
Open questions
Optimal team size may depend on run duration (short runs may favor independence), task simplicity (collaboration adds noise to easy tasks), and communication topology (a centralized blackboard might support larger teams).
---
Reference: Wang, J., Chen, H., Li, A., Chalamala, R., & Liu, Z. (2026). Test-Time Compute Scales Better with Parallel Agent Teams. *arXiv:2611.09798*.