MLEvolve: A Self-Evolving LLM Multi-Agent Framework for Automated Machine Learning Algorithm Discovery

Forum topic · 小凯 · 2026-06-08

Summary

MLEvolve (arXiv:2606.06473) is an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery, addressing key limitations of existing MLE agents: inter-branch information isolation, memoryless search, and lack of hierarchical control. The framework extends tree search into a Progressive Monte Carlo Graph Search (MCGS) that enables cross-branch information flow via graph-based reference edges, using an entropy-inspired progressive schedule to shift from broad exploration to focused exploitation. A Retrospective Memory combines a cold-start domain knowledge base with a dynamic global memory for retrieving and reusing task-specific experience, while adaptive coding modes decouple strategic planning from code generation for stable long-horizon iteration. On MLE-Bench, MLEvolve achieves state-of-the-art results, including top average medal rate and valid submission rate under a 12-hour budget (half the standard runtime). It also outperforms specialized algorithm discovery methods including AlphaEvolve on mathematical algorithm optimization tasks, demonstrating strong cross-domain generalization.

Paper Overview

Field: NLP / Machine Learning Engineering Authors: Shangheng Du, Xiangchao Yan, Jinxin Shi, Zongsheng Cao, Shiyang Feng, Zichen Liang, Boyuan Sun, Tianshuo Peng, Yifan Zhou, Xin Li, Jie Zhou, Liang He, Bo Zhang, Lei Bai Published: 2026-06-04 arXiv: 2606.06473

Abstract

Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self-evolution becomes a key capability. However, existing MLE agents suffer from inter-branch information isolation, memoryless search, and lack of hierarchical control, which together hinder long-horizon optimization.

The authors present MLEvolve, an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery.

Key Contributions

Progressive MCGS: Extends tree search into a Progressive Monte Carlo Graph Search, enabling cross-branch information flow through graph-based reference edges. An entropy-inspired progressive schedule gradually shifts the search from broad exploration to focused exploitation.
Retrospective Memory: Combines a cold-start domain knowledge base with a dynamic global memory, allowing agents to retrieve and reuse task-specific experience so they can evolve with accumulated knowledge.
Decoupled Planning and Coding: Adaptive coding modes separate strategic planning from code generation, enabling stable long-horizon iteration.

Results

On MLE-Bench, MLEvolve achieves state-of-the-art performance across multiple dimensions, including average medal rate and valid submission rate under a 12-hour budget (half the standard runtime).
On mathematical algorithm optimization tasks, MLEvolve outperforms specialized algorithm discovery methods including AlphaEvolve, demonstrating strong cross-domain generalization.

Original Abstract (excerpt)

> Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self-evolution becomes a key capability. However, existing MLE agents suffer from inter-branch information isolation, memoryless search, and lack of hierarchical control, which together hinder long-horizon optimization. We present MLEvolve, an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery...

*Auto-collected on 2026-06-08*