Human children learn math in a well-established sequence: first counting, then addition and subtraction, then multiplication and division, then fractions, then algebra. This is the carefully designed "Common Core" standard built on decades of education research, where each step builds on the previous one.
But what if a language model — trained only to predict the next token — develops mathematical reasoning on its own? Is the order random, or does it also learn counting before addition?
A paper at COLM 2025 offers a striking answer: the developmental order of mathematical capabilities in language models correlates strongly with the human curriculum.
The Experiment: MathCAMPS
The researchers built a synthetic dataset called MathCAMPS. It is not an ordinary math problem bank — it encodes 44 fine-grained mathematical skills spanning grades K-8 (kindergarten through 8th grade).
The skill ordering follows the human Common Core standard, roughly:
1. Counting and comparison: greater than, less than, equal to 2. Addition and subtraction: single-digit, two-digit 3. Multiplication and division: times tables, multi-digit division 4. Fractions: understanding, comparison, operations 5. Basic geometry: shapes, area, perimeter 6. Pre-algebra: unknowns, equations
Every problem is novel — never seen in training data — so the test measures genuine reasoning rather than memorization.
The authors then analyzed multiple open-source LLMs (full checkpoint trajectories from pretraining through post-training) to determine at which training stage each capability emerges.
Core Finding: The Order Is Not Random
During pretraining, the order in which models acquire math skills shows a measurable correlation with the human curriculum.
This is not a vague "kind of similar" — it is a statistically significant correlation. Even when training data is randomly shuffled (so the model does not see counting before addition), the model still spontaneously masters simple skills before complex ones.
More specific findings:
1. The most basic skills appear earliest: counting and comparison are acquired first 2. Fractions and algebra appear late: consistent with human education — fraction understanding requires number sense, algebra requires arithmetic 3. Skills within the same "grade band" tend to emerge together: elementary-level skill clusters appear roughly simultaneously
This was not designed. Nobody told the model "you must learn addition before multiplication." The order emerges spontaneously from the single objective of next-token prediction.
Instruction Tuning: Not All Skills Benefit
The researchers also analyzed how instruction tuning — a widely used post-training method — affects different math skills.
The result is not "everything improves." Some skills do improve, but others degrade.
While the paper does not list a full 44-skill comparison table, the finding itself is crucial: "universal optimization" applied to models may yield positive returns on some abilities and negative returns on others. Moreover, the effect is distributed at a cognitive granularity — not global.
This aligns with recent research showing that post-training optimization trades certain "foundational" abilities for higher "helpfulness" scores.
Explaining the Emergent Order
The paper offers no conclusive causal explanation — that is beyond its experimental scope. But several possibilities are worth considering:
Possibility 1: Natural distribution in the data. Even with shuffled training data, texts about counting and comparison may be intrinsically simpler than texts about fractions and algebra. The model learns simple patterns first, then complex ones.
Possibility 2: Cognitive complexity is intrinsic. Multiplication genuinely is harder than addition — regardless of language or data source. This "difficulty gradient" exists in the structure of mathematics itself, and any learning system must climb it.
Possibility 3: Evidence of emergent structure. Language models develop internal representations through next-token prediction. There may be a correspondence between the hierarchy of these representations and the cognitive hierarchy of mathematical concepts. Fractions depend on addition and multiplication as sub-skills, so the model must first build those representations before constructing fraction representations.
Whichever explanation holds, the finding points to an important realization: language model learning is not random pattern-matching on a blank slate. It is constrained by something intrinsic — perhaps the structure of mathematics itself, or the joint effect of data and optimization — that forces capabilities to emerge in a specific order.
A Reviewer's Take
Among AI research findings, this type of result fascinates me most. Not because it proves "AI resembles humans" — it may actually prove that "mathematics itself has intrinsic structure, and any learning system that extracts patterns from data will eventually advance along that structural line."
Think about it. Multiplication is essentially repeated addition. If you don't understand addition — if you don't know what combining two numbers means — how could you possibly understand multiplication? This is not a design choice of human education systems; it is a logical necessity.
So a language model failing to learn multiplication before addition is not because it "resembles a child." It is because mathematics itself is that way. It is Platonic — the relationships between concepts exist prior to any learning system. You don't need to design a curriculum; logic has already written one.
But the other finding — that instruction tuning harms some skills — deserves caution. When we "align" models, we may inadvertently prune certain capabilities. A model that becomes more "helpful" may simultaneously become worse at fractions. And this is invisible when looking only at aggregate evaluation scores.
This points to a dilemma in AI evaluation: aggregate metrics mask internal changes in capability distribution. Perhaps we need to track model capabilities at fine-grained skill level, the way doctors monitor blood panels.
Reference Paper
Shubhra Mishra, Gabriel Poesia, Noah Goodman. "From Next-Token to Mathematics: The Learning Dynamics of Mathematical Reasoning in Language Models." COLM 2025.