Paper Overview
Field: Machine Learning Authors: Sajad Goudarzi, Samaneh Zamanifard, Moloud Nasiri, Hamed Rahimian Posted: 2026-09-25 arXiv: 2609.27041
Summary
Mathematical reasoning benchmarks are typically organized by topic, but language models may organize their internal computation by reusable reasoning approach instead. This paper investigates whether open math-capable LLMs organize internally by topical sub-skill or by reasoning approach, and presents evidence that the approach is the key organizing axis.
Method
The authors introduce a generation-replay protocol:
1. A model first generates a solution to a math problem. 2. The exact prompt-plus-generation trajectory is replayed. 3. Activation-importance signatures are extracted over the reasoning tokens.
These signatures are clustered without supervision across eight models and five mathematical reasoning sources, and the recovered structure is evaluated with structural, semantic, and intervention tests.
Findings
- Across all 40 model-source cells, recovered clusters outperform size-matched random baselines.
- Two independent frontier LLM judges found approach-level coherence in 77-82% of ground-truth clusters, versus only 6-11% for within-source controls.
- Topic-pure clusters typically received finer-grained labels than the topics themselves.
- In approach-controlled prompts, changing the required reasoning method changed cluster membership in seven of eight model conditions, while paraphrasing largely preserved membership.
Implications
LLMs with math ability organize internal mathematical computation by reasoning approach, not by benchmark topic. Topic-stratified benchmarks and topic-balanced training corpora may therefore miss the truly critical axis: even deliberately topic-balanced corpora can remain unbalanced in reasoning approaches.
--- *Auto-collected on 2026-09-25*