HERA: Experience as a Compass for Self-Evolving Multi-Agent RAG Orchestration
*An editorial explainer based on Li & Ramakrishnan, "Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts" (arXiv:2604.00901)*
Why static orchestration fails
Most multi-agent RAG systems assign fixed roles (retriever, verifier, generator) connected in a preset pipeline. This works for simple factual queries like "What was Einstein's nationality?"—but breaks down on complex multi-hop questions, e.g., asking about the 2023 Nobel Chemistry Prize, how nucleoside-modified mRNA research applies to vaccines, variant effectiveness, and how Moderna, Pfizer, and BioNTech are positioned.
Such queries require cross-domain retrieval, comparison, and synthesis. With static orchestration, pipelines may move to generation before all necessary retrieval happens—producing incomplete answers, redundant searches, or missing steps entirely. As the paper puts it, existing methods "rely on static agent behaviors and fixed orchestration strategies, leading to brittleness on diverse, multi-hop tasks."
The problem is not that agents aren't smart enough—it's that fixed "sheet music" cannot handle jazz-style improvisation.
HERA's core idea: experience as a compass
HERA (Hierarchical Evolution of multi-agent RAG with Role-Aware adaptation) lets the system learn orchestration from experience rather than pre-planning everything. It has two coordinated levels:
Global layer: query-specific agent topology
Instead of a fixed query → retrieval → verification → generation pipeline, HERA:
1. Samples candidate topologies — multiple possible agent connection structures per query. 2. Executes and evaluates them, computing reward from answer quality. 3. Accumulates experience — successful topologies are recorded and preferred for similar future queries. 4. Optimizes the topology-generation policy over time.
The optimization is inspired by GRPO (Group Relative Policy Optimization), lifted from token level to structure level.
Local layer: role-aware prompt evolution
Each agent's role prompt is not static. HERA applies dual-axis adaptation:
- Operational axis (what to do): concrete skills like query decomposition, keyword selection, source relevance evaluation, and when to stop retrieving.
- Behavioral axis (how to act): style choices like exploration vs. exploitation, breadth vs. depth-first retrieval, verbosity, and when to seek help.
- Li, S., & Ramakrishnan, N. Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts. arXiv:2604.00901. https://arxiv.org/abs/2604.00901
A credit assignment mechanism—based on counterfactual/causal reasoning (how much does the result change if agent A is removed or behaves differently?)—identifies which agent behaviors contribute most, reinforcing them in the evolving prompts.
Emergent self-organization
Notably, HERA's topology analysis shows an emergent property: sparse exploration yields compact, high-value multi-agent networks. Rather than exhaustively searching configurations, HERA explores a small, experience-guided subset—and discovers that effective networks are simpler than expected. Useful collaboration patterns emerge rather than being hand-designed.
Experimental results—an honest read
HERA was evaluated on six knowledge-intensive benchmarks with a 38.69% average improvement over recent baselines:
| Benchmark | Result | Note | |---|---|---| | HoVer | +64.95% vs CORAG | Strong gain | | Bamboogle | F1 -5.4% vs ExSearch | Performance drop | | AmbigQA | Best reported performance | Leads | | Three others | Varying improvements | Solid average |
The Bamboogle regression matters: HERA is not universally better, and the average depends heavily on baseline selection. To its credit, the paper reports the failure openly. The durable contributions are not the headline number but the demonstrations that (1) dynamic topologies beat fixed ones, (2) role prompts can usefully evolve from experience, and (3) an experience library offers a viable mechanism for "learning to collaborate."
From design to cultivation
HERA signals a broader shift in multi-agent systems: from *designing* (blueprints, fixed schedules, top-down control) to *cultivating* (providing learning mechanisms and feedback loops, then letting effective patterns emerge). Future implications include less hardcoding, dynamic adaptation as a default, accumulated experience as a core system asset, and embracing emergent behavior within designed boundaries.
Open problems remain: interpretability of evolving topologies (critical for high-stakes domains), computational cost of experience accumulation and topology search, generalization of experience to new domains, and how humans retain oversight of continuously evolving agents.
Takeaway
Real learning comes from experience, not from manuals. HERA's bet: rather than shipping a perfect orchestration design, build a mechanism that lets agents learn how to collaborate. The data says this mostly works—sometimes dramatically—but not always. That honesty is exactly what makes it interesting.