Are Stated Reasoning Steps Causally Load-Bearing? Measuring CoT Faithfulness via Activation Patching

Paper Overview

Field: ML Authors: Abhiram Bhupatiraju, Rayan Nyaupane Published: 2026-09-25 arXiv: 2609.27038

Summary

Chain-of-thought (CoT) monitoring assumes that the reasoning a model writes reflects the computation that directly produces its answer. Previous faithfulness metrics have been predominantly behavioral — editing the reasoning text and observing how the answer changes. This work instead measures faithfulness causally, at the activation level, on self-generated reasoning. Unlike prior causal audits that measure degradation, each intervention carries a known predicted target: every patch should switch the answer to a specific counterfactual entity derivable by construction.

Methodology

• Synthetic multi-hop lookup tasks with 2–6 hops.
• The residual stream is patched at the token span where the model states each intermediate step, using activations from corresponding counterfactual runs.

Key Results

• Qwen3-4B: at the most responsive mid-network layers, 76.9% ± 2.8% of stated steps are causally load-bearing (CLB). Random-position null hypothesis: 11.3%; patching underlying prompt facts: 83% — i.e., stated steps carry ~96% of the attainable effect.
• Behavioral tests overestimate faithfulness: standard behavioral evaluation on the same questions yields 88.2%, overstating causal faithfulness by 11.4 percentage points (question-matched; 111:14 discordant pairs, p < 1e-15), and up to 20 points on the simplest questions.
• Capability dimension: Qwen3-1.7B has substantially lower causal faithfulness overall (54.8%), collapsing as reasoning depth increases (68% at 2 hops, 30% at 6 hops), while Qwen3-4B stays relatively flat.

Conclusion

Although stated reasoning can be causally meaningful, standard behavioral tests tend to overestimate its causal faithfulness — especially on easy samples where the model reasons most fluently.

---

*Auto-collected on 2026-09-25*