Can Small Agents Collaborate to Beat a Single Large Language Model?
- Authors: Agata Żywot, Xinyi Chen, Yifei Yuan, Anders Søgaard, Maarten de Rijke
- Published: 2026-01-16
- Source: https://arxiv.org/abs/2601.11327
- Type: Research paper (Agentic Search)
- Small multi-agent systems beat large single models: On tool-intensive benchmarks, a small orchestrated system outperforms substantially larger single-agent LLMs, even with direct tool access for the large model.
- Orchestrator reasoning matters most: Placing reasoning capability at the orchestrator yields the largest performance gains.
- Sub-agent reasoning can hurt: Enabling reasoning in sub-agents provides limited or even negative benefits.
- Orchestrator capacity drives performance: Overall system performance is driven primarily by orchestrator capacity rather than sub-agent capacity.
- Orchestration over scaling: Improved agentic performance depends more on architectural orchestration than on raw model scaling.
- Factual retrieval
- Multi-hop reasoning
- Scientific question answering
- Mathematical problem solving
- A Systematic Framework for Enterprise Knowledge Retrieval
- Retrieval Augmented Generation and Understanding in Vision: A Survey
- Synergizing RAG and Reasoning: A Systematic Review
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
- Agentic Information Retrieval
TL;DR
Recent progress in language modeling has largely relied on scaling model size, yet larger models do not reliably improve performance on tasks requiring multi-step reasoning and tool use. This paper asks: can well-organized systems built from smaller models outperform much larger language models? Using a minimally designed multi-agent system — a single orchestrator plus a small set of specialized sub-agents with restricted communication — the authors find that small multi-agent systems can outperform substantially larger single-agent models, even when the latter have direct access to tools.
Abstract (original English)
> Recent progress in language modeling has largely relied on scaling model size, yet larger models do not reliably improve performance on tasks requiring multi-step reasoning and tool use. Multi-agent collaboration offers a potential alternative, raising a key question: can well-organized systems built from smaller models outperform much larger language models? We address this question using a minimally designed multi-agent system with a single orchestrator and a small set of specialized sub-agents with restricted communication. On tool-intensive benchmarks spanning factual retrieval, multi-hop reasoning, scientific question answering, and mathematical problem solving, we conduct controlled comparisons between small multi-agent systems and large single-agent models. We find that small multi-agent systems can outperform substantially larger single-agent models, even when the latter have direct access to tools. Reasoning at the orchestrator yields the largest gains, while enabling reasoning in sub-agents provides limited or negative benefits. Overall system performance is driven primarily by orchestrator capacity rather than sub-agent capacity. These results suggest that improved agentic performance depends more on architectural orchestration than on raw model scaling.
Key Findings
Experimental Setup
The evaluation covers tool-intensive benchmarks spanning four task categories:
Controlled comparisons are conducted between small multi-agent systems and large single-agent models. (Specific numerical results should be verified against the original PDF tables.)
Context and Implications
This work sits at the intersection of agentic search and large-scale search/recommendation systems, addressing how to redistribute responsibilities across retrieval, ranking, generation, and tool calling in the LLM era. Implications for the field:
1. Architecture: The agentic paradigm makes retrieval frequency and strategy themselves learnable objects, beyond the classic cascade of retrieval + reranking + generation. 2. Evaluation: Metrics shift from static nDCG toward task success rate, citation accuracy, and multi-hop reasoning chain integrity. 3. Engineering: Latency, cost, interpretability, and safety remain hard constraints for industrial deployment — orchestration of smaller models offers a path to optimize academic benchmarks without prohibitive serving costs.
Limitations
Potential limitations include experiment scale constrained by compute budget, benchmark distributions that may not match real user traffic, English-centric data leaving cross-lingual generalization unknown, and security risks of agent systems operating on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommendation settings.