Scaffolding vs. Skyscrapers: Mathematical Reasoning Is Not Trained by Writing Code

> A review and interpretation of *What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code*

The Bricklayer and the Architect

The post opens with an analogy: Xiao Ming spends three years laying bricks and becomes a master mason; Xiao Hong studies structural mechanics and materials science without laying many walls. Asked to design a skyscraper, only Xiao Hong can produce something earthquake-resistant, functional, and elegant. The paper's thesis: AI training has a similar confusion—reasoning ability comes from training on structured thinking, not from code itself.

The Problem

Paper: Yuze Zhao, Xinran Zhang, Guoqiang Xu, Haonan Wang, Yikang Shen, Yining Ye, Zhezheng Ying, Jiaya Jia, Jian Zhao, Jingdong Wang. *What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code*. arXiv:2605.19762, 2026.

A near-consensus belief in AI holds that adding code to pretraining data improves reasoning. But "code data" is not a single category—it includes pure executable programs, code comments, Code-NL mixes (e.g., Stack Overflow), Math-Code mixes, and Math-Text mixes. If adding code improves reasoning, the true cause might be accompanying technical documentation, structured reasoning traces ("first... then... therefore..."), or math-adjacent content. The paper asks: is it code itself, or something else in the code data?

Experimental Design

The authors ran a large-scale 10T-token pretraining experiment with fine-grained domain separation, building a corpus of cleanly separated buckets:

| Data type | Description | |---------|------| | Pure Code (Code) | Independently executable programs | | Code-NL | Technical documentation with explanations | | Math-Text | Mathematical derivations, theorem proofs | | Math-Text Mix | Textbooks, step-by-step solutions | | Math-Code Mix | Algorithms in math papers | | General | Wikipedia, news, etc. |

They then systematically varied domain proportions and measured task performance.

Three Key Findings

1. Pure code ≠ a general reasoning enhancer. When code is restricted to pure executable programs and Code-NL is strictly controlled, code greatly improves coding ability but does not broadly boost reasoning—and it even *competes* with knowledge-intensive tasks, especially complex mathematical reasoning.

2. The real driver is structured reasoning traces. Cross-domain mixed data (code-text, math-text) containing explicit step-by-step reasoning—"first we observe...", "next, assume...", "therefore..."—is the true textbook for learning to reason, not code syntax.

3. Structured density within the math budget is key. With a fixed math budget, increasing the density of structured math samples yields significant gains on difficult math reasoning while largely preserving coding performance. Better 10B tokens of high-quality step-by-step derivations than 100B tokens of stray code.

Mechanistic Evidence

Routing analyses show that data composition changes are reflected in the model's expert activation patterns. Switching between code and math activates different experts, providing mechanistic evidence of competitive and synergistic cross-domain interactions—the model is not a uniform sponge.

Implications

• From volume to structure: carefully tuning the proportion of structured reasoning signals beats mindlessly scaling code data.
• Scaffolding theory: learning complex skills comes from practicing intermediate structures (the reasoning process), not final forms (writing code)—analogous to practicing scales in music or dissecting argument structure in writing.
• The Feynman parallel: like Feynman's Brazilian students who could recite physics terms without understanding phenomena, a model can "recite" syntactically correct code without learning to think if its training data lacks explicit reasoning structure.

Why This Paper Matters

1. It punctures the "code = reasoning" industry myth with rigorous experiments. 2. It shows the key to reasoning is structured reasoning data, not code volume. 3. It offers concrete data-mixing strategies with direct guidance for training next-generation models.

Reference

Zhao, Y., Zhang, X., Xu, G., Wang, H., Shen, Y., Ye, Y., Ying, Z., Jia, J., Zhao, J., & Wang, J. (2026). *What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code*. arXiv preprint arXiv:2605.19762.

*"Lay ten thousand walls and you remain a bricklayer; understand the mechanics of one wall and you become an architect."*