Imagine two parents: one good at math but weak in language, the other the reverse. Their child—without any schooling—inherits the 'strength modules' of both brains.
That is what Darwin Family does. The 'parents' are Qwen3.5-27B and its reasoning-distilled variant; the 'child' is the merged model—which scores 86.9% on GPQA Diamond (graduate-level scientific reasoning), ranking 6th out of 1,252 models, beating more than a dozen larger, costlier models.
And the entire process requires no training at all. No gradient descent, no backpropagation—only evolutionary algorithms searching and recombining in weight space.
1. The Premise: Reasoning Is Already in Pretraining
The Darwin paper rests on a core hypothesis, supported by a growing body of interpretability research: LLM reasoning ability is formed mainly during pretraining, not during instruction tuning or RLHF.
The paper cites evidence including probing studies since the BERT era showing that different layers encode different language and reasoning functions; causal-intervention experiments showing key functions concentrate in a few neurons; and the success of chain-of-thought prompting, which suggests reasoning mechanisms already exist and just need the right way to be invoked.
The authors ask a sharp question: if reasoning is already encoded in the weights, can we enhance it not by training but by re-arranging the weights?
Answer: yes—using evolutionary algorithms.
2. Method: Model Merging as a Genetic Algorithm
The framework has three key innovations.
① A 14-dimensional 'merge genome.' Each merge strategy is encoded as 14 parameters: global fusion ratio, attention-head weights, FFN weights, embedding weights, sparsification density, independent ratios for six layer blocks, the MRI trust parameter, and the merge-kernel interpolation method—like an organism's DNA.
② MRI-Trust Fusion: diagnostics guiding evolution. The paper proposes Model-layer Response Importance (MRI), a diagnostic that analyzes each tensor's response importance on reasoning tasks to provide a prior signal for merging. The MRI trust parameter τ balances diagnostic signal and evolutionary exploration—evolved τ stably lands at 0.36–0.56, indicating the best strategy combines both. Removing MRI (evolution only) costs 2.5pp on GPQA; using MRI alone without evolution costs 1.3pp.
③ Architecture Mapper: cross-architecture hybridization. It supports 'breeding' between Transformers and Mamba via compatibility scoring and greedy matching across architectures. Darwin-4B-Genesis is a Transformer + Mamba hybrid.
3. Results
Darwin-27B-Opus reaches 86.9% on GPQA Diamond (#6 of 1,252) and improves ARC-Challenge from the parents' 71.0%/74.0% to 77.9%. The whole evolution takes a single H100 and about 5 hours.
A particularly interesting finding: at every scale, evolution consistently chooses to preserve the father model's attention layers while heavily recomposing the reasoning mother's FFN layers. α_attn stabilizes at 0.15–0.32, α_ffn at 0.84–0.93. This is no coincidence—attention handles 'routing' while FFN handles 'computation,' so different roles call for different strategies.
4. Where I'm Not Fully Convinced
- The principle behind cross-architecture merging. The paper claims the Architecture Mapper can map tensors between Transformer attention and Mamba's SSM. But their computation principles are entirely different—context-weighted sum vs. state-space recursion. What does directly interpolating weights even mean? The paper shows quantitative results but no principled explanation. My guess: the 'hybridization' is far milder than it sounds—likely combining partial FFN components rather than true structural recombination.
- Dependence on the MRI calibration set. MRI uses a 123-sample calibration set whose quality directly affects the merge. If a model's reasoning strengths lie outside the calibration set's coverage, MRI could give misleading signals.
- Capability ceiling. The paper honestly admits Darwin cannot create abilities neither parent has. If both parents know nothing about quantum mechanics, merging ten thousand times still won't produce a model that does.
- Title: Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning
- Authors: Taebong Kim, Youngsik Hong, Minsik Kim, Sunyoung Choi, Jaewon Jang, Junghoon Shin, Minseo Kim (VIDRAFT Inc.)
- Preprint: arXiv:2605.14386 (cs.NE)
- Submitted: May 14, 2026 (NeurIPS 2026 submission)
- Core contribution: an evolutionary merging framework with a 14-dim genome plus MRI-guided diagnostics, significantly boosting LLM reasoning with no training. Flagship model hits 86.9% on GPQA Diamond (rank 6/1,252)
- Paper: https://arxiv.org/abs/2605.14386
- Models & code: https://huggingface.co/collections/FINAL-Bench/darwin-family
5. My Take
Darwin Family, like the Valiant paper I covered before, pursues the same goal: improving model capability without additional training. Valiant works on data encoding; Darwin works on weight-space recombination.
If a large model is a library, traditional training is 'building an extension,' while model merging is 'rearranging the existing shelves.' Darwin shows that rearranging alone has enormous potential.
Paper info
1. Kim, T., et al. (2026). Darwin Family. arXiv:2605.14386. 2. Akiba, T., et al. (2025). Evolutionary optimization of model merging recipes. *Nature Machine Intelligence*. 3. Wortsman, M., et al. (2022). Model soups. *ICML 2022*. 4. Yadav, P., et al. (2023). TIES-Merging. *NeurIPS 2023*. 5. Rein, D., et al. (2023). GPQA. arXiv:2311.12022.