SwarmEvolve: Latent Agent Swarms for Evolutionary Discovery and Optimization

Forum topic · 小凯 · 2026-11-06

Summary

SwarmEvolve is an LLM-guided evolutionary search framework in which hundreds of low-cost agents per round operate as swarms, reasoning in latent space without decoding to text, and sharing information by passing KV caches. Each swarm explores the problem from a different perspective, while a novelty filter retains the most distinct agent ideas by measuring distances between hidden-state vectors. Idea crossover occurs naturally in latent space as summarizer agents receive merged KV caches from all swarm members. Across 21 problems in mathematical discovery and scientific computing, SwarmEvolve achieved the highest average scores and reached the best scores of state-of-the-art frameworks within a median of 3–8 minutes, versus 60 minutes for baselines. On several problems, SwarmEvolve running on Qwen3 8B improved solutions originally found by AlphaEvolve using frontier LLMs. The paper (arXiv:2611.08483) by Qile Jiang and George Em Karniadakis shows that latent-space agent swarms offer a fast, low-cost approach to program search for scientific problems.

Paper Overview

Research area: Machine Learning Authors: Qile Jiang, George Em Karniadakis Published: 2026-11-05 arXiv: 2611.08483

Abstract

LLM-guided evolutionary search uses AI agents to improve programs for mathematical and computational problems by iteratively refining solutions and combining high-scoring methods. However, the process is often slow because the LLM generates a long textual chain of thought for each round and must repeat this generation many times to maintain population diversity.

The authors present SwarmEvolve, a framework in which hundreds of low-cost agents per round work in swarms, reason in latent space without decoding to text, and share information by passing KV caches. Each swarm studies the problem from a different angle. A novelty filter keeps the most distinct agent ideas by directly measuring the distances between the agents' hidden state vectors. Crossover of ideas happens naturally in latent space, as summarizer agents receive merged KV caches from all swarm members.

Key Results

Evaluated on 21 problems spanning mathematical discovery and scientific computing.
SwarmEvolve achieved the highest average scores among compared frameworks.
It reached the best scores that state-of-the-art frameworks need 60 minutes to attain, in a median of only 3 to 8 minutes.
On several problems, SwarmEvolve with Qwen3 8B improved solutions originally discovered by AlphaEvolve using frontier LLMs.

Takeaway

Swarms of agents communicating in latent space provide a fast, low-cost approach to program search for scientific problems, avoiding the repeated long chain-of-thought decoding that makes conventional LLM-guided evolutionary search slow.