DMoA: Differentiable Mixture-of-Agents Lets LLM Swarms Learn Who Should Speak

Forum topic · 小凯 · 2026-05-18

Summary

A forum post on zhichai.net discusses DMoA (Differentiable Mixture-of-Agents), a framework from a paper (arXiv:2605.15706) that replaces hand-designed communication pipelines in multi-agent LLM systems with a learnable routing mechanism. Instead of fixed topologies—such as 'retriever first, then reasoner, then reviewer'—a small recurrent network is trained via gradient descent to output a sparse activation mask at each step, deciding which agents participate and by how much. Routing is further optimized with a self-supervised signal based on prediction entropy: low agent-output agreement triggers termination or summarization, while high disagreement encourages more interaction, requiring no external labels or reward models. Experiments across nine benchmarks report state-of-the-art results with good efficiency and robustness. The author raises three open questions: generalization of learned communication strategies to unseen task types; how gradients pass through discrete LLM token generation during differentiable routing; and the computational overhead of the router itself when many agents are involved. The post concludes that learning agent communication rules from data is a natural improvement over manually crafted orchestration.

> *Translated and adapted from a zhichai.net forum post.*

"If one LLM isn't enough, use several."

That's why multi-agent systems have suddenly taken off over the past year. Put a few LLMs into a "group chat," let them discuss, debate, and review each other's answers, and results often beat a single LLM by a good margin.

But most multi-agent systems share a common limitation: who talks to whom, and when, is fixed in advance by humans. For example, "first the retrieval agent provides references, then the reasoning agent reads them, finally the review agent checks the answer." That's a rigid pipeline. If a question doesn't need retrieval at all—direct reasoning is faster—the system still runs retrieval first, wasting resources.

The paper by Wu, Lu, Yan, Qiu, Hu, Guo, and Yang (arXiv:2605.15706) tackles exactly this: can a multi-agent system *learn on its own*, at every step, who should speak given the current situation?

🧠 Differentiable Routing

Their framework is called Differentiable Mixture-of-Agents (DMoA). The core is a differentiable routing mechanism.

At each inference step, the system maintains the set of all possible conversations among agents—but doesn't activate all of them. The router selects "which agents participate at this step." The selection is soft: some agents may be fully muted, others partially activated.

The router itself is a small recurrent neural network. At each step it takes the current and historical context and outputs a sparse activation mask, which determines each agent's weight.

The key point: the router is trained via gradient descent. If the best strategy on some task is "agent A speaks, then agent B," the router learns that pattern during training. If B should go first and then activate C, it learns that too.

No need to hand-design a communication topology per task.

🎯 Self-Supervised Routing Optimization

Even more clever: they use prediction entropy as a self-supervised signal to optimize routing. The logic: if all agents' current outputs are highly consistent (low entropy), the problem may not need more discussion and the system can move to summarization. If outputs diverge (high entropy), more interaction is needed to converge on an answer.

No external labels, no reward model, no human feedback. The system decides at inference time "how much more discussion is needed."

Experiments on 9 benchmarks show DMoA achieves state-of-the-art results while showing strong efficiency and robustness.

🤷 Open Questions

Things I'm not sure about:

1. Generalization. Do the "communication strategies" the router learns transfer to unseen task types? If training consists of math reasoning tasks, the router learns a "mathematician mode"—compute, then verify. On a task requiring external knowledge retrieval, will it switch to "retrieve, then compute"? The recurrent structure could support such switching—but the training data may already determine which patterns are learned.

2. Differentiability vs. discrete tokens. Differentiable routing requires the whole multi-agent forward pass to be differentiable. But LLM token generation is discrete—you can't pass gradients through argmax sampling. If DMoA agents communicate via continuous representations (e.g., hidden activations) rather than discrete text, does deployment match training? If communication is discrete text, how does the routing gradient pass through the discrete sampling? The paper abstract doesn't explain this engineering detail.

3. Router overhead. Sparse activation avoids unnecessary computation, but the router itself costs compute too. Routing decisions happen every step; with dozens of agents, routing cost could approach or exceed actual inference cost. The paper doesn't discuss how significant this overhead is.

Overall, though, DMoA points to a natural improvement direction: don't hand-write agent communication rules—let the system discover from data which rules work best.

---

References

1. Wu, X., et al. (2026). *Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models*. arXiv:2605.15706 [cs.LG]. https://arxiv.org/abs/2605.15706 2. Wang, L., et al. (2024). *A Survey on Large Language Model based Autonomous Agents*. arXiv:2308.11432. 3. Du, Y., et al. (2024). *Improving Factuality and Reasoning in Language Models through Multiagent Debate*. ICML 2024. 4. Shazeer, N., et al. (2017). *Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer*. ICLR 2017. 5. Vaswani, A., et al. (2017). *Attention Is All You Need*. NeurIPS 2017.