MLEvolve: Self-Evolving Multi-Agent Framework Tops MLE-Bench with Half the Compute Time

Forum topic · 小凯 · 2026-06-07

Summary

MLEvolve is a self-evolving multi-agent framework from Shanghai AI Laboratory that achieved top results on MLE-Bench, a benchmark of 75 Kaggle-style machine learning engineering tasks. Using only 12 hours per task (half the 24-hour budget of competitors), MLEvolve reached a 65.3% medal rate, 34.7% gold medal rate, and a 100% valid submission rate, outperforming systems like MARS+ and AIBuildAI that use stronger or newer models. Its design combines three innovations: (1) Progressive Monte Carlo Graph Search, which upgrades tree search to a graph with reference edges enabling cross-branch knowledge sharing, entropy-inspired soft exploration-exploitation scheduling, and stagnation detection; (2) Retrospective Memory, a cold-start knowledge base plus dynamic global memory with BM25+FAISS hybrid retrieval and phase-aware querying, requiring no extra reflection LLM calls; and (3) hierarchical planning with adaptive code generation across full, stepwise, and diff modes. On 15 AlphaEvolve mathematical optimization tasks, MLEvolve matched or beat Google DeepMind's AlphaEvolve on 14. Code is open-sourced on GitHub.

MLEvolve: A Self-Evolving Multi-Agent Framework for Automated ML Engineering

Overview

MLEvolve, a self-evolving multi-agent framework from Shanghai AI Laboratory, topped MLE-Bench (75 Kaggle competition tasks) with a 65.3% medal rate, 34.7% gold rate, and 100% valid submissions — using only 12 hours per task, half the budget of competitors. It also matched or beat Google DeepMind's AlphaEvolve on 14 of 15 mathematical optimization tasks.

Why existing ML engineering agents fall short

1. Information silos — tree search branches never share discoveries; branch A's feature-engineering trick never reaches branch B. 2. Goldfish memory — each planning decision starts from scratch; only scalar rewards are passed along. 3. Full rewrites every time — regenerating entire solutions regardless of state, breaking working code.

MLEvolve's three core mechanisms

1. Progressive Monte Carlo Graph Search (MCGS)

Replaces the tree with a graph using reference edges: nodes can reference successful nodes in other branches.
Four expansion modes: base expansion; within-branch evolution (review last k attempts); cross-branch reference (pull global Top-N solutions); multi-branch aggregation (fuse complementary insights).
Progressive scheduling: high-entropy exploration early, elite-guided exploitation later, via a soft entropy-inspired probabilistic switch between UCT and greedy selection.
Stagnation detection: no local progress triggers within-branch evolution; no global improvement triggers multi-branch aggregation.

2. Retrospective Memory

Cold-start knowledge base: per-task-type domain knowledge (model choices, usage guides).
Dynamic global memory: automatically logs plans, results, analyses, and feedback — no extra LLM reflection calls.
Hybrid retrieval: BM25 keyword + FAISS semantic search, fused with Reciprocal Rank Fusion.
Phase-aware retrieval: planning stage queries with the initial plan; debugging stage queries with error messages to find similar solved issues.

3. Hierarchical planning + adaptive code generation

A Planner decides *which* module to change and why; a Coder executes at the appropriate granularity:
Base: full code from scratch (initial phase)
Stepwise: module-by-module generation (complex pipelines)
Diff: precise local edits (fine-tuning a working solution)

Results

MLE-Bench (75 Kaggle tasks)

| Method | Runtime | Medal rate | Gold rate | Valid submissions | |--------|---------|-----------|-----------|-------------------| | AIDE | 24h | 17.1% | 9.4% | 82.8% | | ML-Master 2.0 | 24h | 56.4% | 19.6% | 95.6% | | MARS+ | 24h | 62.7% | 33.8% | 100% | | AIBuildAI | 24h | 63.1% | 25.8% | 100% | | MLEvolve | 12h | 65.3% | 34.7% | 100% |

Mathematical optimization (15 AlphaEvolve tasks)

MLEvolve matched or exceeded AlphaEvolve on 14/15 tasks. Highlights:

Hex packing: 3.928476 vs 3.930092 (lower is better — MLEvolve wins)
Autocorrelation 3rd (v): 1.458770 vs 1.468762 (lower is better — MLEvolve wins decisively)
Sum-diff 1: 1.190177 vs 1.147989 (higher is better — MLEvolve wins)
Autocorrelation 2nd: 0.905422 vs 0.896280 (higher is better — MLEvolve wins)
Only loss: Kissing number d11: 592 vs 593 (AlphaEvolve wins)

Engineering insights

1. Graph search > tree search in open search spaces — reference edges let good discoveries propagate across branches. 2. Memory without reflection — automatically accumulated structured records plus hybrid retrieval beat extra LLM reflection calls at lower cost. 3. Adaptive code-generation granularity — choosing base/stepwise/diff based on state matters more than any single choice. 4. General frameworks can beat specialized systems — a generic search + memory + hierarchical generation paradigm outperformed the purpose-built AlphaEvolve on math tasks.

Availability

Paper: https://arxiv.org/abs/2606.06473
Code: https://github.com/InternScience/MLEvolve
Project page: https://internscience.github.io/MLEvolve

Configuration: Gemini-3.1-Pro-preview, 500 expansion steps per task, 12-hour runtime, single H200 GPU.

Closing thoughts

The most exciting aspect is the self-evolving paradigm: the agent accumulates experience, adjusts strategy, and fuses insights from divergent directions, emerging with solutions beyond any single path — mirroring how human research actually works. The natural next question: what if MLEvolve used itself to optimize its own search strategy?