[论文] MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Le...

论文概要

研究领域: NLP 作者: Shangheng Du, Jinxin Shi, Shuaiyu Zhang, Yiming Wu, Junpeng Zhao, Shiyang Feng, Xin Li, Liang He, Xiangchao Yan, Bo Zhang 发布时间: 2026-10-15 arXiv: 2610.22344

中文摘要

大型语言模型正从辅助科学研究扩展到自主开发和改进 AI 系统，推动 AI4AI 作为递归自我改进的途径。然而，每个实验的反馈反映了多个先前决策，长的 MLE 运行包含许多昂贵实验。训练因此必须从长交互轨迹中选择有用的监督并从中学习。我们提出一种后训练方法并应用于训练 MLExpert——一个 35B 混合专家智能体模型。我们设计 Kaggle-MLE Factory 构建可执行任务和模型导向实验的交互框架。分层智能体 SFT 通过结果引导的监督将通用工具使用和编码能力转移到 MLE。窗口交互 RL（WIRL）将连续实验结果之间的多轮决策视为执行窗口并分配共享的结果奖励。在 MLE-Bench Lite 上，MLExpert 将基础模型的平均奖牌率从 37.9% 提高到 72.7%，达到与 Kimi-K3 和 GLM-5.3 等大型前沿模型相当的性能。

原文摘要

Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI). Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback. However, feedback from each experiment reflects several preceding decisions, and a long MLE run contains many such costly experiments. Training must therefore select useful supervision from long interaction trajectories...

#论文 #arXiv #NLP #小凯