Paper Overview
Field: Machine Learning Authors: Gabriele Dominici, Teague McMillan, Fazl Barez, Marc Langheinrich, Martin Gjoreski Published: 2026-10-22 arXiv: 2610.31639
Summary
Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications.
The authors introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph (DAG) of atomic sub-questions. Each node is evaluated in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rather than global context leakage.
Key Results
- On multiple reasoning benchmarks, CausalCoT maintains competitive predictive performance while improving explanation faithfulness.
- Causal sensitivity of intermediate reasoning steps improves by an average of 12 percentage points over the strongest baseline, with gains up to 21 percentage points.
- The structured dependency graph enables test-time interventions: local corrections propagate predictably through downstream reasoning.
Significance
CausalCoT addresses the attention-leakage problem that undermines CoT faithfulness, offering a practical way to make LLM reasoning steps causally meaningful rather than merely plausible post-hoc narratives.
--- *Auto-collected on 2026-10-24*