Paper Overview
Research area: Machine Learning Authors: Sajad Goudarzi, Samaneh Zamanifard, Seyed Amin Seyed Haeri, Moloud Nasiri, Hamed Rahimian Published: 2026-10-05 arXiv: 2610.00331
Summary
When selecting mathematical training data for LLMs, a natural organizing principle is topic: probability examples for probability targets. An alternative is reasoning approach: worked solutions that share a solution method with the target, even when the mathematical domain differs. The paper asks which relation produces greater transfer after fine-tuning.
Experimental Design
The authors evaluate two counterbalanced 2×2 designs:
- Design 1: probability and combinatorics, crossed with invariant reasoning and double counting (2,000 problems)
- Design 2: number theory and geometry, crossed with complement and pigeonhole reasoning (800 problems)
- Same-approach (SA) sources share the target's solution method but change the topic
- Same-topic (ST) sources share the topic but change the method
- SA outperformed ST in all 40 pooled seed comparisons.
- Main design: model-level advantages of 8.2–16.2 percentage points (mean 10.8).
- Second design: 12.0–16.0 percentage points (mean 14.3).
- All ten model-level 95% confidence intervals excluded zero.
- In both designs, ST sources were *more* similar to targets under embedding and lexical metrics — the SA advantage runs counter to measured statement-level similarity rankings.
In each design, every cell serves as the held-out target in turn:
Each source appears once in each role, so additive source-quality effects cancel out in the equally-weighted aggregate comparisons. Five base models × three seeds per design were tested.
Key Findings
Conclusion
These findings confirm that, for the combinations evaluated, reasoning approach is a more effective matching criterion than topic for mathematical transfer in LLM fine-tuning.