MLEvolve: A Self-Evolving Multi-Agent Framework for Automated Machine Learning Algorithm Discovery

Forum topic · 小凯 · 2026-06-08

Summary

MLEvolve is an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery, presented in arXiv paper 2606.06473. It addresses three weaknesses of existing MLE agents: inter-branch information isolation, memoryless search, and lack of hierarchical control. The framework extends tree search to Progressive MCGS, using graph-based reference edges for cross-branch information flow and an entropy-inspired progressive schedule that shifts search from broad exploration to focused exploitation. A Retrospective Memory combines a cold-start domain knowledge base with a dynamic global memory for retrieving and reusing task-specific experience, while adaptive coding modes decouple strategic planning from code generation for stable long-horizon iteration. On MLE-Bench, MLEvolve achieves state-of-the-art results across multiple metrics, including average medal rate and valid submission rate under a 12-hour budget—half the standard runtime. It also outperforms specialized algorithm-discovery methods, including AlphaEvolve, on mathematical algorithm optimization tasks, demonstrating strong cross-domain generalization.

Paper Overview

Field: NLP Authors: Shangheng Du, Xiangchao Yan, Jinxin Shi, Zongsheng Cao, Shiyang Feng, Zichen Liang, Boyuan Sun, Tianshuo Peng, Yifan Zhou, Xin Li, Jie Zhou, Liang He, Bo Zhang, Lei Bai Published: 2026-06-04 arXiv: 2606.06473

Abstract

Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self-evolution becomes a key capability. However, existing MLE agents suffer from inter-branch information isolation, memoryless search, and lack of hierarchical control, which together hinder long-horizon optimization. We present MLEvolve, an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery.

Key Contributions

Progressive MCGS: Extends tree search so that cross-branch information flows through graph-based reference edges, and an entropy-inspired progressive schedule gradually shifts the search from broad exploration to focused exploitation.
Retrospective Memory: Combines a cold-start domain knowledge base with a dynamic global memory, enabling retrieval and reuse of task-specific experience so the agent evolves with accumulated experience. (Truncated in source.)
Hierarchical control: Adaptive coding modes decouple strategic planning from code generation for stable long-horizon iteration.

Results

On MLE-Bench, MLEvolve achieves state-of-the-art performance across multiple dimensions, including average medal rate and valid submission rate under a 12-hour budget (half the standard runtime).
On mathematical algorithm optimization tasks, MLEvolve outperforms specialized algorithm-discovery methods including AlphaEvolve, showing strong cross-domain generalization.

--- *Auto-collected on 2026-06-08*