KisMATH Study: Do LLMs Truly Reason or Just Memorize? Causal Chain-of-Thought Analysis

Forum topic · 小凯 · 2026-05-11

Summary

KisMATH (arXiv:2507.11408, accepted to TACL 2026) by researchers from ISI Kolkata, IRIT, and LINAGORA Labs investigates whether chain-of-thought (CoT) in large language models constitutes genuine causal reasoning or merely decorative approximate retrieval. The team introduces Causal CoT Graphs (CCGraphs)—directed acyclic graphs automatically extracted from LLM-generated reasoning traces, with math expressions as nodes and causal dependencies as edges—built from 1,671 problems across GSM8K, MATH500, and AIME. Using attention suppression on 15 open-source LLMs (1B–70B parameters, including Gemma 3, Qwen 3, DeepSeek-R1, and Llama 3), they show that suppressing reasoning nodes significantly increases answer entropy (p < 10⁻¹²), proving reasoning nodes are causal mediators of answers. Additionally, models assign significantly higher probability to the longest causal reasoning paths (R-paths) than to random alternatives, indicating LLMs internally implement CCGraph-like implicit structures. The study reconciles prior conflicting findings: random number perturbation preserves structural scaffolding, so performance is unaffected, while structure-aligned intervention collapses reasoning—showing structure, not specific values, is the key mechanism behind CoT.

Overview

> Paper: *KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?* > Authors: Soumadeep Saha, Akshay Chaturvedi, Saptarshi Saha, Utpal Garain, Nicholas Asher > Institutions: ISI Kolkata, IRIT, LINAGORA Labs > Published: Accepted to TACL, 2026 > arXiv: 2507.11408 > Dataset & code: https://kismath.github.io

When DeepSeek-R1 or OpenAI o3 produces a long "Let me think step by step...", is a reasoning mind at work, or a statistical language machine imitating the *posture* of thinking? The field has split into two camps:

Reasoning Hypothesis: CoT works by decomposing problems into subtasks and solving them step by step (OpenAI o1/o3, DeepSeek-R1).
Approximate Retrieval Hypothesis: CoT is merely approximate retrieval from latent memory; reasoning traces are decorative. Evidence includes insensitivity to 50% random number substitution and pseudo-rewards improving performance (Kambhampati, 2024).

KisMATH offers new empirical evidence for this debate.

Methodology: Causal CoT Graphs (CCGraphs)

Prior tests relied on random perturbation (replacing numbers, shuffling examples), which can destroy values while preserving structural scaffolding—making it impossible to distinguish value-insensitivity from structural independence. KisMATH proposes interventions aligned with the implicit structure of reasoning.

A CCGraph is a DAG automatically extracted from LLM-generated reasoning traces:

Nodes: mathematical expressions (numbers, formulas, operation results)
Edges: fine-grained causal dependencies

Extraction works backward from the answer node, matching expressions via exact string match or shared parse-tree nodes, recursively expanding to question nodes, then reversing edges to form a DAG and pruning orphans. Manual correction was needed for only ~10% of cases (88/40K nodes, 71/300K edges).

From each graph, the R-path (longest Q→A path) is extracted: top-5 longest paths for GSM8K, top-10 for MATH500/AIME; each problem has 9–40 reasoning nodes and 6–10 reasoning hops.

Dataset and Models

| Dataset | Problems | Type | Difficulty | |---------|----------|------|------------| | GSM8K | 983 | Grade-school word problems | ⭐⭐ | | MATH500 | 384 | Olympiad-style math | ⭐⭐⭐⭐ | | AIME | 304 | Competition math | ⭐⭐⭐⭐⭐ |

CoT traces were generated with OpenAI o3-2025-04-16 (5-shot CoT, only correct-answer traces kept). Fifteen open-source LLMs (1B–70B) were tested: Gemma 3 (1B/12B/27B), Qwen 3 (1.7B/8B/32B), DeepSeek-R1 (1.5B/8B/32B/70B), Llama 3.1/3.3 (8B/70B), Qwen 2.5/2.5-Math (7B), DeepSeek-R1 Distill (7B/8B). Compute: 4×A100, ~3000 GPU-hours.

Key Experiments and Findings

1. Reasoning nodes are causal mediators

Using attention suppression—zeroing attention contributions of all tokens belonging to reasoning nodes in the CCGraph—the authors measured answer entropy:

Suppression caused a significant increase in answer entropy (p < 10⁻¹²), with high Kolmogorov distance between distributions.
Conclusion: reasoning nodes causally mediate answers—directly refuting the extreme "CoT is decoration" view.

2. R-paths have special causal status

Suppressing only nodes along the R-path (not all reasoning nodes) still significantly increased answer entropy (high D_KS, very low p-values on GSM8K). Not all reasoning tokens are equally important—nodes on structured paths carry special causal weight.

3. LLMs internally implement CCGraph-like structures

The authors compared the probability of R-paths, P(R) = Π P(r̂(i_δ) | x_{<T_δ}), against random same-length paths avoiding CCGraph nodes:

A significant peak at the 100th percentile across all 15 models on all three splits (only Llama 3.3 70B on AIME was an exception).
Conclusion: LLMs assign significantly higher probability to causal R-paths—they possess implicit knowledge of reasoning structure.

Reconciling Conflicting Conclusions

| Approach | Finding | Interpretation | |----------|---------|----------------| | Random perturbation (50% digit swap) | Performance unaffected | Robust to *values* | | Structure-aligned suppression (this work) | Uncertainty spikes | Sensitive to *structure* |

Unified hypothesis: LLMs use both structure and memory. Random digit substitution preserves the structural framework (which operations, in what order), so the model can still "follow the map"; destroying the structure itself (suppressing key reasoning nodes) leaves the model lost—much like a human mathematician who can swap specific numbers in a proof but cannot complete it with proof steps removed.

Advantages Over Prior Work

Compared with Tan (2023, 27 manually annotated GSM8K graphs), Lee et al. (2025, 30 graphs), and Bogdan et al. (2025, 10 problems), KisMATH offers:

1. Automation — SymPy parsing + graph traversal, no manual annotation 2. Fine granularity — expression-level nodes, not sentence-level 3. Causal precision — strict attention-suppression interventions 4. Scale — 1,671 samples, 15 models, 3 difficulty tiers

Limitations and Future Directions

Limitations: math-only coverage (no code/logic/science reasoning); ~10% manual fixes for LaTeX errors; graphs capture only causal dependencies (no planning/backtracking/reflection edges); open-source models only; static graphs extracted from complete traces.

Future work: extend to programming and scientific reasoning; dynamic CCGraphs updated during token generation; richer edge types; downstream applications in error localization, RL reward design, and structured reasoning visualization.

Conclusion

KisMATH provides the most rigorous empirical evidence to date on whether LLMs truly reason:

1. LLMs perform causal reasoning: suppressing reasoning nodes significantly increases answer uncertainty (p < 10⁻¹²). 2. LLMs encode implicit structure: R-path probabilities significantly exceed random-path baselines. 3. Structure is key, values are secondary: random perturbation preserves structure and hence performance; structural intervention collapses reasoning.

The study does not fully refute approximate retrieval—LLMs likely exploit both structure and memory. But it establishes that structured reasoning is a genuine mechanism behind CoT gains, not pure decoration. What we see when DeepSeek-R1 "thinks" is neither human-level understanding nor rote recitation, but an intermediate cognitive phenomenon we are only beginning to understand.

References

1. Saha S, et al. *KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?* arXiv:2507.11408, accepted to TACL, 2026. 2. Wei J, et al. Chain-of-thought prompting elicits reasoning in large language models. *NeurIPS* 2022. 3. Guo D, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv:2501.12948, 2025. 4. Kambhampati S. LLMs can't plan, but can help planning. *SSRN*, 2024. 5. Lanham T, et al. Measuring faithfulness in chain-of-thought reasoning. arXiv:2307.13748, 2023.