Scaffolding vs. Skyscrapers: Mathematical Reasoning Isn't Trained by Writing Code
> *A reading of "What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code"*
The Analogy: Architect vs. Bricklayer
Imagine two architecture students. Xiao Ming spends three years on construction sites hauling bricks—he can lay a perfect wall. Xiao Hong spends a year studying structural mechanics, materials science, and architectural history—she understands *why* walls are built the way they are. Ask each to design a skyscraper: Xiao Ming may be faster at laying local walls, but Xiao Hong can design a building that survives earthquakes, looks good, and functions well.
This paper addresses the AI-training version of this problem: we assumed that having AI write more code makes it reason better, but the truth is that reasoning ability comes from training on structured thinking—not from code itself.
The Problem
Paper: Yuze Zhao, Xinran Zhang, Guoqiang Xu, Haonan Wang, Yikang Shen, Yining Ye, Zhezheng Ying, Jiaya Jia, Jian Zhao, Jingdong Wang. *What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code*. arXiv:2605.19762, 2026.
Field: AI / large language models / mathematical reasoning / data engineering
A near-consensus belief in AI circles holds that adding code to pretraining data makes models better at reasoning. The rationale sounds plausible: code is logically rigorous and structured, writing it requires step-by-step derivation, and strong models like GPT-4 and Claude were indeed trained on massive code corpora. So companies scraped GitHub and Stack Overflow, treating code as "fertilizer for reasoning."
But the paper asks a crucial, overlooked question:
> "Is it code itself that improves reasoning, or something *else* in code data?"
Background: The Black Box of Pretraining Data
- Pretraining means having the model "read" massive internet text, learning from statistical patterns without an explicit curriculum.
- "Code data" is not a single category: pure executable code (Python, Java, C++), code comments, Code-NL mixes (Stack Overflow Q&A), math-code mixes (algorithms in papers), and math-text mixes (textbooks, derivations).
- Confounding variables: code data usually comes bundled with technical documentation, structured reasoning traces ("first... then... therefore..."), and math content. These *accompaniments* may be the true source of reasoning gains.
Core Experimental Design
Rather than a naive "code vs. no code" comparison, the authors ran large-scale pretraining experiments on a 10T-token corpus with fine-grained domain separation:
| Data type | Description | |---------|------| | Pure code | Independently executable programs | | Code-NL | Technical documents with explanations | | Math-Text | Mathematical derivations, theorem proofs | | Math-Text Mix | Math textbooks, step-by-step solutions | | Math-Code Mix | Algorithms in math papers | | General | Wikipedia, news, etc. |
They systematically varied domain proportions and measured performance across tasks.
Three Paradigm-Shifting Findings
Finding 1: Pure code ≠ a general reasoning enhancer
> When code is restricted to pure executable programs and Code-NL data is strictly controlled—code substantially improves programming ability but does not act as a general reasoning enhancer.
Feeding a model more Python makes it better at Python, but not better at solving math or logic problems. Worse, code training even competes with knowledge-intensive tasks—especially complex mathematical reasoning.
Finding 2: The real driver is structured reasoning traces
What made "code-trained" models *seem* better at reasoning? Cross-domain structured reasoning traces—code-text and math-text mixed data, not executable code itself. These mixtures share a common feature: explicitly unfolded reasoning processes ("first we observe...", "next, assume...", "therefore..."). This explicit reasoning scaffolding—not code syntax—is the true textbook for learning to reason.
Finding 3: Structured density within the math domain is key
> Within a fixed math budget, increasing the density of structured math samples yields significant gains on difficult math reasoning while largely preserving coding performance.
Instead of 100B tokens of scattered code, 10B tokens of high-quality step-by-step math derivations work better for math reasoning—without harming other capabilities.
Mechanistic Evidence: Expert Activation Patterns
Routing analyses show that data-composition changes are reflected in the model's expert activation patterns. When the model "switches" between code and math, different experts (subnetworks) activate—providing mechanistic evidence of competitive and synergistic interactions between cross-domain capabilities. The model is not a uniform sponge; code training and math training compete for specialized regions.
Implications: Data-Centric Optimization
1. From volume to structure: Carefully tuning the proportion of structured reasoning signals beats mindlessly scaling code data. 2. The "scaffolding" theory: Learning complex skills comes from practicing intermediate structures (the reasoning process), not just final products (writing code)—like practicing scales before improvising, or dissecting an essay's logic before writing one. 3. The Feynman perspective: Feynman found Brazilian students could fluently recite physics terms without understanding phenomena. Similarly, a model can "recite" code (generate syntactically valid programs), but without explicit reasoning structure in training data, it hasn't truly learned to think—a superb bricklayer, not an architect.
Why This Paper Matters
1. Pops the bubble: Rigorous experiments overturn the "code = reasoning" industry myth. 2. Points the way: Improving reasoning requires structured reasoning data, not code volume. 3. Practical value: It provides concrete data-mixing strategies with direct guidance for training next-generation models.
Reference
Zhao, Y., Zhang, X., Xu, G., Wang, H., Shen, Y., Ye, Y., Ying, Z., Jia, J., Zhao, J., & Wang, J. (2026). *What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code*. arXiv preprint arXiv:2605.19762.
---
*Lay ten thousand walls and you remain a bricklayer; understand the mechanics of one wall and you become an architect.*