Can Small Agents Collaborate to Beat a Single Large Language Model?
Paper: arXiv:2601.11327 Authors: Agata Żywot, Xinyi Chen, Yifei Yuan, Anders Søgaard, Maarten de Rijke Published: 2026-01-16 | Area: Agentic Search
Overview
Recent progress in language modeling has largely relied on scaling model size, yet larger models do not reliably improve performance on tasks requiring multi-step reasoning and tool use. This work asks a key question: can well-organized systems built from smaller models outperform much larger language models?
Method
The authors use a minimally designed multi-agent system consisting of:
- A single orchestrator agent
- A small set of specialized sub-agents with restricted communication
- Small multi-agent systems can outperform substantially larger single-agent models, even when the larger models have direct access to tools.
- Reasoning at the orchestrator yields the largest gains.
- Enabling reasoning in sub-agents provides limited or negative benefits.
- Overall system performance is driven primarily by orchestrator capacity rather than sub-agent capacity.
- A Systematic Framework for Enterprise Knowledge Retrieval
- Retrieval Augmented Generation and Understanding in Vision: A Survey
- Synergizing RAG and Reasoning: A Systematic Review
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
- Agentic Information Retrieval
They conduct controlled comparisons between these small multi-agent systems and large single-agent models on tool-intensive benchmarks covering factual retrieval, multi-hop reasoning, scientific question answering, and mathematical problem solving.
Key Findings
Implications
The results suggest that improved agentic performance depends more on architectural orchestration than on raw model scaling. For search, recommendation, and RAG practitioners, this points to an efficiency-oriented design path: invest reasoning budget in the orchestrator, keep sub-agents lightweight and specialized, and treat coordination structure as a first-class design variable rather than defaulting to a single massive model.
Caveats
Specific quantitative results should be verified against the original PDF. As with much of this research area, evaluation relies on academic benchmarks, and gap to real-world latency, cost, and safety constraints remains an open problem.
Original Abstract
> Recent progress in language modeling has largely relied on scaling model size, yet larger models do not reliably improve performance on tasks requiring multi-step reasoning and tool use. Multi-agent collaboration offers a potential alternative, raising a key question: can well-organized systems built from smaller models outperform much larger language models? We address this question using a minimally designed multi-agent system with a single orchestrator and a small set of specialized sub-agents with restricted communication. On tool-intensive benchmarks spanning factual retrieval, multi-hop reasoning, scientific question answering, and mathematical problem solving, we conduct controlled comparisons between small multi-agent systems and large single-agent models. We find that small multi-agent systems can outperform substantially larger single-agent models, even when the latter have direct access to tools. Reasoning at the orchestrator yields the largest gains, while enabling reasoning in sub-agents provides limited or negative benefits. Overall system performance is driven primarily by orchestrator capacity rather than sub-agent capacity. These results suggest that improved agentic performance depends more on architectural orchestration than on raw model scaling.