Can Small Agents Collaborate to Beat a Single Large Language Model?
- Authors / Affiliations: Agata Żywot, Xinyi Chen, Yifei Yuan, Anders Søgaard, Maarten de Rijke
- Published: 2026-01-16
- Source: https://arxiv.org/abs/2601.11327
- Type: Academic paper (Agentic Search)
- A single orchestrator responsible for planning and coordination
- A small set of specialized sub-agents with restricted communication
- Small multi-agent systems can outperform substantially larger single-agent models, even when the larger models have direct access to tools.
- Reasoning at the orchestrator yields the largest gains; enabling reasoning in sub-agents provides limited or negative benefits.
- Overall system performance is driven primarily by orchestrator capacity rather than sub-agent capacity.
- These results suggest that improved agentic performance depends more on architectural orchestration than on raw model scaling.
- Original paper: *Can Small Agents Collaborate to Beat a Single Large Language Model?* — https://arxiv.org/abs/2601.11327
Overview
Recent progress in language modeling has largely relied on scaling model size, yet larger models do not reliably improve performance on tasks requiring multi-step reasoning and tool use. Multi-agent collaboration offers a potential alternative, raising a key question: can well-organized systems built from smaller models outperform much larger language models?
Method
The paper addresses this question using a minimally designed multi-agent system with:
Controlled comparisons are performed between small multi-agent systems and large single-agent models on tool-intensive benchmarks covering factual retrieval, multi-hop reasoning, scientific question answering, and mathematical problem solving.
Key Findings
Implications for Search, Recommendation, and Personalization
1. Architecture: Agentic paradigms treat retrieval frequency and strategy as learnable decision objects, complementing the classic cascade of retrieval → rerank → generation. 2. Evaluation: Offline metrics increasingly diverge from online satisfaction; LLM-as-judge should be cross-validated with human evaluation. 3. Productization: Latency, cost, interpretability, and safety remain hard constraints for industrial deployment. 4. Efficiency: Well-designed orchestration of small models may offer a cost-effective alternative to scaling up a single model.
Limitations and Future Work
Potential limitations include experimental scale constrained by GPU budgets, benchmark distributions that may not match real users, English-centric data limiting cross-lingual generalization, and security risks of agent systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommender systems.
Original Abstract
> Recent progress in language modeling has largely relied on scaling model size, yet larger models do not reliably improve performance on tasks requiring multi-step reasoning and tool use. Multi-agent collaboration offers a potential alternative, raising a key question: can well-organized systems built from smaller models outperform much larger language models? We address this question using a minimally designed multi-agent system with a single orchestrator and a small set of specialized sub-agents with restricted communication. On tool-intensive benchmarks spanning factual retrieval, multi-hop reasoning, scientific question answering, and mathematical problem solving, we conduct controlled comparisons between small multi-agent systems and large single-agent models. We find that small multi-agent systems can outperform substantially larger single-agent models, even when the latter have direct access to tools. Reasoning at the orchestrator yields the largest gains, while enabling reasoning in sub-agents provides limited or negative benefits. Overall system performance is driven primarily by orchestrator capacity rather than sub-agent capacity. These results suggest that improved agentic performance depends more on architectural orchestration than on raw model scaling.