How AI 'Learns' Math: From Next-Token Prediction to Solving Equations
Feynman was famously unable to answer how he learned to think about physics—thinking emerges gradually and mysteriously. A paper from Stanford's Noah Goodman group tackles a more precise version of this question: how do AI models 'learn' mathematics?
The Setup: MathCAMPS
Large language models only do one thing during pretraining: predict the next token, billions of times. Yet somehow they end up able to do arithmetic, solve equations, and reason geometrically.
The researchers built the first systematic experiment to study this:
- They used the US K-8 Common Core standards as a "coordinate axis" of math ability—44 fine-grained skills layered by grade, from counting to solving linear equation systems.
- They created MathCAMPS, a synthetic dataset where each skill has dedicated problems: purely synthetic (never in training data), testing single skills, with consistent formats across skills.
- They evaluated intermediate checkpoints of open-source models, tracking when each capability emerges across training tokens.
- Learned early: counting, number recognition, basic arithmetic, shape/pattern recognition.
- Learned later: multi-step word problems, fractions, algebraic thinking, geometric proofs—skills requiring chains of intermediate states.
- Hardest: inverse reasoning ("given the answer, find the condition"). This mirrors cognitive development: children develop forward reasoning before backward reasoning.
- Improved: addition/subtraction, basic fractions, single-step word problems.
- Degraded: multi-step reasoning, geometric spatial imagination, inequality derivations.
- AI diagnostics: curriculum-aligned tools like MathCAMPS can measure "what grade" a model has reached.
- Training strategy: protect deep reasoning during fine-tuning rather than diluting it with drill-style data.
- Theory: intelligence—biological or artificial—may obey universal learning laws independent of implementation.
Key Finding: AI Learns Math in Grade-Order
The order in which LLMs acquire math skills correlates significantly with the grade-level ordering of human curricula.
Models learn counting and shape recognition (kindergarten) first, then addition/subtraction (grade 1), multiplication/division (grade 3), and finally equations and geometric proofs (middle school).
Crucially: the training data was randomly shuffled. Nobody told the model what to learn first. Yet a learning order self-organized within the model's parameters—one that matches what human educators designed. It's like feeding a child books of random difficulty and watching them still progress from letters → sentences → paragraphs → chapters.
What Emerges First, What Comes Last
Instruction Tuning Is a Double-Edged Sword
Comparing pretraining and instruction-tuned stages:
In short, instruction tuning strengthens pattern matching but weakens deep reasoning—a familiar tension between standardized drilling and genuine mathematical thinking.
Why Might This Happen?
The paper proposes candidate mechanisms (not a complete theory):
1. Natural stratification by complexity: skills with lower computational complexity converge first, no curriculum needed. 2. Different training-signal density: addition is reinforced across almost all numeric text; linear equations appear only in narrow contexts. 3. Hierarchical representation learning: simple features stabilize in shallow layers before relational/conditional reasoning.