[论文] Causal Chain of Thought: Mitigating Attention Leakage via Context Isol...

论文概要

研究领域: ML
作者: Gabriele Dominici, Teague McMillan, Fazl Barez, Marc Langheinrich, Martin Gjoreski
发布时间: 2026-10-22
arXiv: 2610.31639

中文摘要

思维链（CoT）提示可以引出逐步推理论证，看似解释了模型的预测，但这些步骤是否在因果上驱动最终答案并无保证。在标准自回归生成中，稠密注意力允许每个推理步骤关注完整上下文，使模型可以通过直接查找得出答案，同时生成听起来合理但实为事后辩护的论证。我们提出 CausalCoT——一个通过将问题分解为原子子问题的有向无环图（DAG）来强制实现因果透明性的推理框架。每个节点在仅包含其直接因果父节点的隔离上下文中求值。这种马尔可夫结构移除了通往非父节点内容的注意力通路，使下游预测对局部推理而非全局上下文泄露敏感。在多种推理基准测试中，CausalCoT 在保持有竞争力预测性能的同时改善了解释忠实度，中间推理步骤的因果敏感性相比最强基线平均提升 12 个百分点，最高提升 21 个百分点。结构化的依赖图还实现了测试时干预——局部修正可以可预测地传播到下游推理。

原文摘要

Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications. We introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph of atomic sub-questions, evaluating each node in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rath...

自动采集于 2026-10-24

#论文 #arXiv #ML #小凯