A post on zhichai.net discusses a Stanford research paper asking a deceptively simple question: how does a large language model that only predicts the next token end up able to solve equations? The author frames the discussion with a Richard Feynman anecdote about how thinking forms gradually and mysteriously, then walks through the paper's findings.
The Experiment
Researchers borrowed the US Common Core K-8 mathematics standards—44 fine-grained skills spanning kindergarten counting to eighth-grade linear equations—and built a synthetic benchmark called MathCAMPS. Each skill has dedicated synthetic problems (not present in training data), each testing one specific skill. They then evaluated intermediate checkpoints of open-source models on MathCAMPS, tracking when each mathematical capability appears during training.
Key Findings
- Spontaneous curriculum order: Despite training data being randomly shuffled—with no curriculum design—the order in which models acquire math skills shows significant positive correlation with the human grade-level sequence. Counting and shape recognition come first, then addition/subtraction, multiplication/division, and finally equations and geometry proofs.
- Early skills: counting, number recognition, basic arithmetic, shape/pattern recognition.
- Late skills: multi-step word problems, fractions, algebraic thinking, geometric proofs—skills requiring chained reasoning with multiple intermediate states.
- Hardest skills: reverse (bidirectional) reasoning, e.g., deriving conditions from a known answer. This mirrors cognitive development in children, where forward reasoning matures before backward reasoning.
- Improved: addition/subtraction, basic fractions, single-step word problems.
- Degraded: multi-step reasoning (especially long logic chains), geometric spatial reasoning, inequality derivations.
- AI evaluation: tools like MathCAMPS can diagnose what "grade level" a model has effectively reached, analogous to standardized tests for children.
- Training strategy: pretraining could focus on deep reasoning, while fine-tuning should protect it from being diluted by pattern-matching drills.
- Theory: intelligence—biological or artificial—may be governed by universal learning laws independent of implementation.
- Title: From Next-Token to Mathematics: The Learning Dynamics of Mathematical Reasoning in Language Models
- Authors: Shubhra Mishra, Gabriel Poesia, Noah Goodman (Stanford)
- Venue: COLM 2025
- Link: https://openreview.net/forum?id=bJ9aARjtBu
- Dataset: MathCAMPS — 44 skills, K-8 Common Core
Instruction Fine-Tuning Is a Double-Edged Sword
Comparing pretraining and instruction-tuning stages, the study finds:
In other words, fine-tuning strengthens pattern-matching (recognizing common question formats) while weakening deep reasoning—a finding the author compares to the education debate between rote drilling and deep problem-solving.
Proposed Explanations
The paper offers candidate mechanisms rather than a complete theory:
1. Natural complexity stratification—skills converge in order of intrinsic complexity, no curriculum needed. 2. Signal density—basic skills like addition are reinforced across nearly all numeric text, while solving linear equations appears only in narrow contexts. 3. Hierarchical representation learning—neural networks naturally stabilize simple features (magnitude, shape) before complex relational ones.
Implications
The post closes with a Feynman-style reflection praising the paper's precision ( pinpointing roughly which training token count corresponds to acquiring addition, fractions, or equations) and its counterintuitive warning that overly standardized instruction can erode genuine reasoning ability.