> Paper: Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks > arXiv: 2606.29082 | Young-Jun Lee et al. (University of Minnesota) > Dataset: Finch Collection (156K trajectories, 10 domains, 371 tasks)
The Problem with Existing Methods: Capability Lives in the Scaffolding, Not the Model
LLM + evolutionary search has achieved breakthroughs in GPU kernel design, mathematical conjecture proving, and scientific law discovery. But there is a fundamental flaw:
The evolutionary capability lives in the scaffolding, not in the model.
Every new problem starts from scratch; search experience is discarded once used. The model itself cannot iteratively optimize, decide what to mutate, or know when to backtrack — all of that is provided by the external search framework.
EFT asks a deeper question: what if the model learns to evolve itself?
The Core Idea: Evolutionary Trajectories as Supervision
Instead of searching from scratch at test time, the model practices evolution during training.
Training Data: Finch Collection
- 156K evolutionary search trajectories
- 10 domains: math, competitive programming, GPU kernels, scientific law discovery, combinatorial puzzles...
- 371 optimization tasks
- Generated by running the OpenEvolve framework with Qwen3.5-397B; erroneous trajectories filtered out, keeping high-quality samples
- Identify problems in the current solution
- Decide where and how to mutate
- Evaluate improvement directions
- Know when to backtrack
- Circle-packing task: matches SOTA level
- Erdős minimum overlap problem: surpasses base-model counterparts
- Paper: https://arxiv.org/abs/2606.29082
- Authors: Young-Jun Lee, Seungone Kim et al. (University of Minnesota)
Training Paradigm
Search trajectories are converted into supervised signals, teaching 2B–9B open-source models to:
This is not teaching "the answer to this problem," but "how to find the answer."
Experimental Results: Small Models Can Discover
Cross-Task Generalization (22 held-out tasks)
EFT fine-tuned models improve 10.22% on average over base models.
The key is not absolute scores but cross-domain transfer — the model transfers recommender-system optimization strategies to competitive programming, and GPU kernel design experience to math conjectures. This emergent transfer suggests the model has internalized a general "discovery intuition."
Combining with Test-Time RL
EFT models + test-time reinforcement learning:
EFT does not replace search frameworks — it turns the model into a better search participant. It can be embedded directly into existing frameworks as a mutation operator (frozen weights), or adapted further with test-time learning.
Why This Matters
1. Lower barriers — no need for closed-source giants (GPT-4, Claude); 2B–9B open-source models can do high-quality evolutionary search
2. Internalized capability — evolutionary search becomes a model instinct rather than an external tool; new tasks no longer start from zero
3. Reproducible dataset — Finch Collection is public: 156K trajectories across 371 tasks for follow-up research
4. Orthogonal to existing methods — EFT models can plug into any search framework or stack with test-time RL
An Analogy
Traditional methods hand a student (the model) a fresh dictionary (the search framework) for every exam — the student can't actually use it, only read aloud from it.
EFT has the student first spend time practicing how to look things up, take notes, and summarize patterns. On exam day the dictionary is still there, but the student already knows what to look up, how, and when to try a different approach.