"One LLM isn't enough? Then use several."
That's why multi-agent systems have surged in popularity over the past year. Put several LLMs into a "group chat," let them discuss, debate, and review each other's answers, and the results are often notably better than a single LLM.
But most multi-agent systems share a common limitation: who talks to whom, and when, is fixed in advance by humans. For example, "first the retrieval agent provides reference material, then the reasoning agent reads it, finally the review agent checks the answer." That's a fixed pipeline. If a problem doesn't need retrieval at all—direct reasoning is faster—the system still runs retrieval first, wasting resources.
The paper by Wu, Lu, Yan, Qiu, Hu, Guo, and Yang (arXiv:2605.15706) tackles exactly this problem: can a multi-agent system learn, on its own, to decide at every step "who should speak given the current situation?"
🧠 Differentiable Routing
Their framework is called Differentiable Mixture-of-Agents—DMoA. The core is a differentiable routing mechanism.
At each inference step, the system maintains the set of all possible conversations among agents. But it doesn't activate all of them—the router selects "which agents participate at this step." The selection isn't hard: some agents may be fully muted, others partially activated.
The router itself is a small recurrent neural network. At each step it takes the current and historical context and outputs a sparse activation mask, which determines each agent's weight.
The key point: this router is trained via gradient descent. If the best policy on some task is agent A speaks, then agent B, the router learns that pattern during training. If it should be B first, then activate C, it learns that too.
No need to hand-design a communication topology for each task.
🎯 Self-Supervised Routing Optimization
Even cleverer: they use "prediction entropy" as a self-supervised signal to optimize routing. The idea: if all agents' current outputs are highly consistent (low entropy), the problem probably doesn't need more discussion and the system can move to summarization. If outputs diverge heavily (high entropy), more interaction is needed to converge on an answer.
No external labels, no reward model, no human feedback. The system decides at inference time "how long to keep discussing."
Experiments on 9 benchmarks show DMoA achieves state-of-the-art results while demonstrating strong efficiency and robustness.
🤷 What Remains Unclear
Things I'm not sure about:
First, can the "communication strategies" learned by the router generalize to unseen task types? If the training set is all math reasoning, the router learns a "mathematician mode"—compute, then verify. Put it on a task requiring external knowledge retrieval—will it automatically switch to a "retrieve first, then compute" mode? The router's recurrent architecture could in principle make such switches—but its training data may already determine which patterns were ever learned.
Second, differentiable routing requires the entire multi-agent forward pass to be differentiable. But LLM discrete token generation is not differentiable—you can't pass gradients through argmax sampling. If DMoA communicates via continuous representations (e.g., hidden activations) rather than discrete text, does it behave consistently at deployment as in training? If it uses discrete text communication, how do the router's gradients flow through discrete sampling? The abstract doesn't explain this engineering detail.
Third, sparse activation avoids unnecessary computation, but the router itself consumes compute. Routing decisions may be needed at every step; with many agents—say dozens—routing cost could approach or even exceed actual inference cost. The paper doesn't discuss how significant this overhead is.
Overall, though, DMoA offers a natural direction for improvement: don't hand-write the agents' communication rules—let the system discover from data which rules work best.
---
References
1. Wu, X., et al. (2026). *Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models*. arXiv:2605.15706 [cs.LG]. https://arxiv.org/abs/2605.15706 2. Wang, L., et al. (2024). *A Survey on Large Language Model based Autonomous Agents*. arXiv:2308.11432. 3. Du, Y., et al. (2024). *Improving Factuality and Reasoning in Language Models through Multiagent Debate*. ICML 2024. 4. Shazeer, N., et al. (2017). *Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer*. ICLR 2017. 5. Vaswani, A., et al. (2017). *Attention Is All You Need*. NeurIPS 2017.