When AI Learns to Run Its Own Experiments: How MLExpert Gets a 35B Model to Outscore Kimi-K3 on Kaggle
The Scenario
Imagine you're a Kaggle competitor. It's 2 AM, you've just run your first model: score 0.58. Not great. You start wondering: change feature engineering? Change model architecture? Tune hyperparameters? Or rethink from scratch?
Every choice depends on the previous experiment's result. You run an experiment, check the score, decide the next step. This process can last 12 hours and 50 iterations.
Now hand that whole process to AI.
Not a one-shot code generation—that's too easy. Instead, the AI itself decides: when to switch direction, when to backtrack, when to stay the course. 12 hours, 3,000 interaction steps, fully autonomous.
This is the problem MLExpert, just released by Shanghai AI Lab, tackles. The result? A 35B-parameter model achieves a 72.7% medal rate on MLE-Bench Lite, matching larger models like Kimi-K3 and GLM-5.3.
The Core Problem: Credit Assignment in Long-Horizon Optimization
To understand why MLExpert matters, you need to understand the core tension it addresses.
Existing MLE agents (AIDE, MLE-STAR, R&D-Agent) mostly use external orchestration: a predefined workflow controls the experiment sequence, and the LLM only generates or debugs code at designated steps. It's like giving the AI a flowchart to follow.
The problem: real ML engineering doesn't follow flowcharts. You might run three experiments, realize the direction is wrong, and backtrack to experiment one. These decisions require global understanding of the entire experiment history, not just the current step.
Recent work (Arbor, Argus) has shifted toward model-led mode: the model decides what to do next. But this creates a fundamental training difficulty—
One experiment's result is caused jointly by several earlier rounds of decisions.
You changed features (decision A), swapped models (decision B), tuned the learning rate (decision C), then ran an experiment. The score comes out. Who deserves credit—A, B, or C?
This is the credit assignment problem in long-horizon optimization. Standard GRPO gives a terminal reward to the whole trajectory, but an MLE trajectory can span 3,000 steps and dozens of experiments. Terminal rewards simply can't distinguish "good decisions at step 99" from "a bad decision abandoned at step 10."
WIRL: Slicing Credit by Execution Windows
MLExpert's core innovation is WIRL (Windowed Interaction RL). The idea is strikingly simple:
Use each experiment result as a boundary to segment the trajectory.
All model decisions between two experiment results form an "execution window." Every decision in that window shares the same reward—the result of that experiment.
It's like splitting a marathon into sprints. Each sprint has its own finish line and its own result. You don't need to wait 12 hours to know how you're doing; feedback arrives after each segment.
But there's a subtle design choice: the reward isn't a simple binary good/bad, but hierarchical:
- Execution error: -1.0
- No scorable submission: -0.6
- Submitted but below medal threshold: -0.4 + trend score
- Bronze/silver medal: 1.0 + trend score
- Gold medal: 2.0 + trend score
- NatureBench Lite (reproducing Nature-family paper results): base Qwen3.6's Surpass-SOTA rate rises from 0% to 50%, matching GLM-5.3.
- AIME Agent Scaffold optimization: optimizing GPT-4.1-nano's Python scaffold, held-out accuracy improves from a 23.3% base to 43.7%, near Kimi-K3's 40.0%.
- MLS-Bench-Lite (algorithm component optimization): across 12 cross-domain tasks, average verifier reward improves from 0.322 to 0.425, with gains on 10/12 tasks.
- Data layer: Kaggle-MLE Factory auto-constructs training tasks
- Training layer: hierarchical SFT + WIRL learn from execution feedback
- Inference layer: the Interaction Harness enables model-led long-horizon optimization
- GitHub: https://github.com/InternScience/MLExpert
- HuggingFace: https://huggingface.co/InternScience/MLExpert
- Paper: https://arxiv.org/abs/2610.22344
The trend score \(T_m\) is the finishing touch. It compares the current submission's score with the last valid submission: improvement +0.2, flat 0, regression -0.2. This means even without a medal, if scores are steadily rising, the model learns "this direction is right."
This solves a key problem: MLE score scales are not uniform. Image-classification accuracy and NLP F1 can't be compared directly. But "direction of progress" is comparable. The trend score unifies heterogeneous task rewards into one framework.
Asynchronous Collection and Policy Correction
Two engineering details deserve attention.
Asynchronous collection: experiment runtimes vary (some 5 minutes, some 2 hours). WIRL doesn't wait for the full trajectory; each completed execution window joins the training batch. A 3,000-step trajectory may therefore span several policy updates—later decisions use the updated policy, earlier ones the old policy.
Policy correction: this creates an off-policy problem. WIRL's solution is to record each token's log-probability at collection time (rollout policy) and correct with the ratio to the current policy's log-probability. Tokens outside the \([q_{min}, q_{max}]\) interval are excluded from gradient updates.
An analogy: you're writing a long novel, and at chapter 20 the editor says "chapters 1–5 need a style change." You don't rewrite the whole book; you mark which passages to adjust and which to keep. WIRL's policy correction is this kind of "local revision"—keep the long trajectories already collected, only correct the parts skewed by policy drift.
Kaggle-MLE Factory: Turning Competitions into Training Data
Where does training data come from? The team built a Kaggle-MLE Factory—a multi-agent pipeline converting historical Kaggle competitions into executable tasks:
1. Discovery Agent: filters competitions by status, size, submission format 2. Analysis Agent: extracts data structure, fields, modality features 3. Generation Agent: prepares data, builds held-out test sets, implements evaluators 4. Validation Agent: checks data integrity, submission format, evaluator behavior; iteratively fixes issues
Each final task package contains a public workspace, a hidden test set, and a unified evaluation interface. After manual review to exclude tasks with label or scoring problems, they're ready for trajectory collection.
The Factory itself is an example of "AI making data for AI"—using LLM agents to automatically construct training tasks for LLM agents.
Hierarchical Agentic SFT: From General to Specialized
Before WIRL, MLExpert goes through two SFT stages:
Stage 1: general tool use and coding—training basic tool-calling and code-editing on diverse agentic tasks.
Stage 2: MLE domain adaptation—using teacher-model trajectories on Kaggle tasks to train algorithm design, experiment analysis, and iterative optimization.
There's also an elegant result-oriented supervision selection mechanism. Long-horizon trajectories mix effective exploration and failure recovery, but may contain garbage segments where performance degrades after a strong solution is reached. The approach: find the first best submission in a trajectory and measure subsequent degradation. If degradation exceeds a threshold, keep only the portion up to the best submission; otherwise keep everything.
Cut the degraded tail. This echoes the idea that "redundancy comes after the answer": reasoning models overthink mainly in the verification phase after an answer appears, and MLE trajectories degrade mainly in continued attempts after the best solution. Both are "redundancy after the answer."
The Numbers
MLE-Bench Lite main results:
| Model | Med+ | Gold | Any Medal | |-------|------|------|-----------| | Qwen3.6-35B-A3B (base) | 62.1 | 22.7 | 37.9 | | Nex-N2.5-mini | 66.7 | 24.2 | 47.0 | | DeepSeek-V4-Pro Preview | 86.4 | 40.9 | 63.6 | | GLM-5.3 | 90.9 | 59.1 | 72.7 | | Kimi-K3 | 95.5 | 63.6 | 72.7 | | GPT-5.6 Sol | 90.9 | 59.1 | 77.3 | | MLExpert (Ours) | 84.8 | 50.0 | 72.7 |
A 35B model matching Kimi-K3 and GLM-5.3. The advantage holds under the Claude Code general harness too (69.7% vs base 42.4%).
Ablations:
| Training stage | Any Medal | |---------------|-----------| | Base | 37.9 | | + General agent SFT | 47.0 | | + MLE SFT | 62.1 | | + WIRL | 72.7 |
Every stage contributes. WIRL adds 10.6 points on top of SFT.
RL method comparison:
| Method | Any Medal | |--------|-----------| | GRPO | 56.1 | | PPO-style | 60.6 | | WIRL w/o trend reward | 66.7 | | WIRL w/o policy correction | 65.2 | | WIRL | 72.7 |
GRPO and PPO-style both fall below MLE-SFT (62.1%), confirming that terminal-reward credit assignment fails. Removing the trend reward drops WIRL to 66.7%; removing policy correction drops it to 65.2%—both designs are essential.
Generalization: Not Just Kaggle
Do MLExpert's optimization skills transfer?
MLExpert didn't learn "how to run Kaggle"; it learned "how to iteratively optimize an experimental system." That capability is task-agnostic.
Engineering Insights
1. Execution windows are MLE's natural credit-assignment boundary. The only trustworthy feedback point in MLE is the experiment result. Packing decisions between results into windows with shared rewards solves credit assignment without waiting for trajectory end. The idea extends to any long-horizon task with intermediate checkpoints—coding (each test run is a window), science (each measurement), even writing (each chapter review).
2. The trend reward unifies heterogeneous reward scales. Different competitions use AUC, F1, RMSE... but "direction of progress" is universal. The simple \(T_m \in \{-0.2, 0, 0.2\}\) lets the model learn "which direction is right" across tasks.
3. Result-oriented supervision selection beats keeping everything. Long trajectories mix good and bad; keeping it all teaches bad habits. Cutting degraded tails is a simple, effective data-cleaning strategy.
4. Model-led beats external orchestration. MLExpert's Interaction Harness leaves branching, backtracking, and direction switching to the model itself. The harness only provides tools (execute_code, update_answer, analyze, invalidate_node) and state management (an experiment tree, solution nodes).
Closing Thoughts
MLExpert suggests a larger framework: a three-layer AI4AI architecture.
Together these form a closed loop of "AI generating its own data and training itself." Far from true RSI (recursive self-improvement)—MLExpert improves ML engineering ability, not the model itself—but a milestone on the AI4AI path.
WIRL also resonates with several other concepts: "chunk-level attribution GRPO" decomposes terminal rewards to memory-operation granularity, just as WIRL decomposes them to execution-window granularity—both densify sparse terminal rewards. And across papers, the same underlying structure keeps appearing: in long-horizon tasks, terminal rewards are too sparse; you must find trustworthy intermediate feedback points and organize credit assignment around them. MLExpert's feedback point is the experiment result. Other domains have their own: test pass rates for code, measurements for science, chapter reviews for writing. WIRL's framework generalizes.
Open Source
One-sentence summary: MLExpert segments long-horizon MLE trajectories by execution windows, letting a 35B model match Kimi-K3 on Kaggle competitions. The core insight: experiment results are MLE's natural credit-assignment boundary, trend rewards unify heterogeneous reward scales, and cutting degraded tails beats keeping everything—a significant step on the AI4AI path.