MLEvolve: Self-Evolving Multi-Agent Framework Tops MLE-Bench in Half the Time

MLEvolve, a self-evolving multi-agent framework from Shanghai AI Laboratory, recently topped the MLE-Bench leaderboard — and beat Google DeepMind's AlphaEvolve on 14 of 15 mathematical optimization tasks.

Imagine being a Kaggle competitor facing 75 competition tasks, from image classification to tabular regression, with 12 hours on the clock. Your opponents get 24 hours, stronger models, and teams at the OpenAI/DeepMind level. The result: MLEvolve achieved a 65.3% medal rate, 34.7% gold rate, and 100% valid submissions — first place overall.

Why Existing ML Engineering Agents Fall Short

Current MLE agents suffer from three critical weaknesses:

1. Information silos — Each tree-search branch is an island. A clever feature-engineering trick discovered in branch A never reaches branch B. 2. Goldfish memory — Every planning decision starts from scratch. At iteration 50, the agent doesn't remember what failed at iteration 3 or what worked at iteration 17. Search frameworks only pass scalar rewards. 3. Full-rewrite coding — Regardless of state, the agent rewrites the entire solution each time, like rebuilding a whole frontend project just to change a CSS color.

MLEvolve's Three Key Innovations

1. Progressive Monte Carlo Graph Search (MCGS)

Instead of a tree, MLEvolve uses a graph. Beyond parent-child edges, nodes can form reference edges to other branches, letting stuck branches borrow successful ideas from elsewhere:

• Base expansion: generate from the parent node without references
• Within-branch evolution: reflect on the last k attempts in the same branch
• Cross-branch reference: pull the global Top-N solutions as references when stuck
• Multi-branch aggregation: merge complementary insights from multiple strong solutions

A progressive exploration schedule softly shifts from high-entropy exploration early on to elite-guided exploitation later, using an entropy-inspired probabilistic switching between UCT exploration and elite guidance. Stagnation detection triggers within-branch evolution when a branch stalls, and multi-branch aggregation when the global best stops improving.

2. Retrospective Memory

The agent carries an "experience disk" with two parts:

• Cold-start knowledge base: domain knowledge organized by task type (image classification, NLP, tabular regression, etc.), including suitable models and usage guides.
• Dynamic global memory: automatically records plans, results, analyses, and feedback signals after each valid execution. Retrieval combines BM25 keyword matching with FAISS semantic search, fused via Reciprocal Rank Fusion.

Phase-aware retrieval uses the initial plan as a query during planning (to surface relevant successes/failures) and error messages during debugging (to surface similar solved errors). No extra LLM reflection calls — memory accumulates automatically at zero additional cost.

3. Hierarchical Planning + Adaptive Code Generation

A Planner reasons at the module level (what to change and why), while a Coder executes at the code level using one of three granularities:

• Base mode: full code generation from scratch — for initial phases
• Stepwise mode: module-by-module generation — for complex multi-stage pipelines
• Diff mode: precise local edits — for fine-tuning working solutions

Results

MLE-Bench (75 Kaggle tasks)

| Method | Runtime | Medal Rate | Gold Rate | Valid Submissions | |--------|---------|-----------|-----------|-------------------| | AIDE | 24h | 17.1% | 9.4% | 82.8% | | ML-Master 2.0 | 24h | 56.4% | 19.6% | 95.6% | | MARS+ | 24h | 62.7% | 33.8% | 100% | | AIBuildAI | 24h | 63.1% | 25.8% | 100% | | MLEvolve | 12h | 65.3% | 34.7% | 100% |

Half the time, beating every opponent — including AIBuildAI (Claude Opus 4.6) and MARS+ (Gemini-3-Pro).

Mathematical Optimization (15 AlphaEvolve tasks)

MLEvolve matched or beat AlphaEvolve on 14 of 15 tasks, e.g.:

• Hex packing: 3.928476 vs 3.930092 (lower is better — MLEvolve wins)
• Autocorrelation 3rd (v): 1.458770 vs 1.468762 (lower is better — MLEvolve wins big)
• Sum-diff 1: 1.190177 vs 1.147989 (higher is better — MLEvolve wins)
• Autocorrelation 2nd: 0.905422 vs 0.896280 (higher is better — MLEvolve wins)

The only loss: Kissing number d11: 592 vs 593.

Engineering Insights

1. Graph search beats tree search — in open search spaces, allowing cross-branch information flow is a qualitative change; good inventions propagate automatically. 2. Memory doesn't need reflection — automatically accumulated structured records + hybrid retrieval suffice, saving reflection token costs. 3. Code generation granularity should be adaptive — the key is automatically choosing the right mode based on current state. 4. A general framework can beat specialized systems — a good search paradigm outlasts task-specific tricks.

Open Source

Code: https://github.com/InternScience/MLEvolve

Built on Gemini-3.1-Pro-preview, with a 500-step expansion cap per task, 12-hour runtime, and a single H200 GPU.

Closing Thoughts

The most exciting aspect is the self-evolving paradigm: an agent that accumulates experience, adjusts strategies, and fuses insights across directions during search, ultimately producing solutions that transcend any single path. It mirrors human research — reading others' work (cross-branch reference), reviewing one's failures (within-branch evolution), combining ideas across fields (multi-branch aggregation), and switching between exploration and exploitation (progressive scheduling).

The next interesting question: what if MLEvolve used itself to optimize its own search strategy? That would be true recursive evolution.

---

Paper: MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery Code: github.com/InternScience/MLEvolve Project page: internscience.github.io/MLEvolve