EvoMAS: Evolutionary Algorithms Automatically Design Multi-Agent Systems for LLMs

Forum topic · 小凯 · 2026-06-29

Summary

EvoMAS reframes multi-agent system (MAS) design as configuration generation rather than code generation. Instead of having LLMs write fragile MAS code or relying on rigid human templates, EvoMAS encodes each MAS as a structured configuration covering agent roles, backbone models, system prompts, tool access, and communication topology. An evolutionary loop—Select, Mutate, Crossover, Consolidate—searches this configuration space, with LLM-driven feedback-conditioned mutations and an experience memory that stores successful evolution strategies. On benchmarks, EvoMAS reportedly reaches 58.7% on BBEH versus 46.2% for the best human-designed baseline, 63.8% on SWE-Bench-Verified with Qwen3-235B (and 79.1% with Claude-4.5-Sonnet), and outperforms EvoAgent on WorkBench. Key findings include benefits from heterogeneous model assignment per agent role, growing gains on complex tasks, higher executability than code-generation approaches, and continued improvement as evolution compute scales. Limitations include high evaluation cost, coverage of only cooperative agents, and reliance on LLM-as-judge reward signals.

Overview

Designing multi-agent systems (MAS) by hand is like assembling a Swiss watch — every gear and spring placed by human intuition. EvoMAS proposes letting evolution be the architect: instead of generating code or following fixed templates, it treats MAS design as configuration generation, then applies evolutionary search over that configuration space.

Why existing approaches fall short

Code generation is fragile: LLM-written MAS code frequently fails due to syntax errors, API changes, and environment issues; debugging costs can exceed hand-writing.
Templates are rigid: Fixed role structures (e.g., MetaGPT's product manager + architect + engineer) limit the expressible design space and adapt poorly across tasks.
Human design doesn't scale: Hand-crafted architectures (Peer Review, MetaGPT, ADAS) degrade sharply when the model or task changes.

EvoMAS's insight: don't generate code, generate configuration. Config is a declarative description layer; a lightweight runtime interprets it, so configurations can vary freely without execution failures.

The configuration: a "genome" for MAS

Each MAS is encoded along five dimensions:

| Dimension | Description | Example | |---|---|---| | Agent Roles | Role definition for each agent | "code reviewer", "test engineer" | | Backbone Models | LLM powering each agent | Claude-4.5-Sonnet, Qwen3-235B | | System Prompts | Per-agent system prompts | "You are a strict code reviewer..." | | Tool Access | Tools callable by each agent | file I/O, search, code execution | | Communication Topology | Inter-agent structure | star, chain, fully connected, hierarchical |

The evolutionary loop: Select → Mutate → Crossover → Consolidate

1. Select: Retrieve the k most relevant candidate configurations from the pool, based on task metadata similarity and historical performance. 2. Mutate: Make targeted, feedback-driven modifications — adjust a prompt, swap a backbone model, change tool permissions, or rewire topology edges — guided by execution feedback rather than random changes. 3. Crossover: Recombine two parent configurations, e.g., inheriting one's topology and the other's efficient agent designs. 4. Consolidate: Summarize successful evolution trajectories into a compact experience memory. Crucially, it stores *which modification strategies worked* — a form of meta-learning that guides future search.

Results

| Benchmark | EvoMAS | Best human-designed baseline | Gap | |---|---|---|---| | BBEH (reasoning) | 58.7% | 46.2% (Peer Review + Qwen3-235B) | +12.5 pp | | SWE-Bench-Verified | 63.8% | 44.3% (MetaGPT + Qwen3-235B) | +19.5 pp | | SWE-Bench-Verified (Claude-4.5-Sonnet) | 79.1% | — | matches leaderboard top | | WorkBench (tool use) | beats EvoAgent | — | +7.1 points | | BBEH (vs EvoAgent) | beats EvoAgent | — | +10.5 points |

Key findings

No single model wins everywhere: EvoMAS assigns different models to different roles (e.g., Claude for architecture, Qwen for code generation); heterogeneous combinations outperform single-model setups.
Gains grow with task complexity: On simple tasks EvoMAS is close to a single agent; on complex tasks like software engineering the gap widens dramatically.
Higher executability: Config generation yields far higher successful execution rates than code generation, since a runtime interpreter guarantees correctness.
Compute scaling: More evolution steps mean continued improvement, while other methods plateau — a natural advantage of population-based search.

Why evolution fits MAS design

1. The configuration space is combinatorially explosive (agents × models × prompts × topologies × tools); gradient descent can't handle this discrete space, but population search can. 2. Task success is a sparse, delayed reward; EvoMAS uses LLM-as-judge as a proxy to convert it into intermediate feedback. 3. Experience memory enables cumulative learning rather than reasoning from scratch each time. 4. EvoMAS aligns with test-time compute scaling — allocating compute to structural search rather than deeper single-agent reasoning.

Analogy: automatic breeding for MAS

| Biological evolution | EvoMAS | |---|---| | Genome | MAS configuration (roles, models, prompts, topology) | | Gene pool | Configuration pool (human designs + accumulated history) | | Natural selection | Task success rate as fitness | | Mutation | LLM-driven feedback-conditioned edits | | Hybridization | Configuration recombination | | Species memory | Experience memory from Consolidate | | Adaptive radiation | Different tasks evolve different MAS forms |

Limitations and future directions

Limitations

Evolution requires many evaluations, so compute cost exceeds single-shot inference.
Only cooperative agents are considered; adversarial settings are not covered.
Relies on LLM-as-judge, so reward signals may be biased.

Future directions

Finer-grained heterogeneous model evolution (e.g., MoE-style routing).
Online/real-time evolution rather than per-task evolution.
Cross-task transfer via the Consolidate memory mechanism.

References

Yuntong Hu, Yuting Zhang, Matthew Trager, Yi Zhang, Shuo Yang, Wei Xia, Stefano Soatto. *Evolutionary Generation of Multi-Agent Systems*. ICML 2026. arXiv:2602.06511