CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate

Authors: Finn G. Vamosi & Nils D. Forkert — University of Calgary Paper: arXiv:2511.22854 | Code: https://github.com/finnvamosi/CRAwDAD

Background and Motivation

Causal reasoning is a core human cognitive ability but remains challenging for large language models (LLMs), which often behave as "causal parrots" — repeating correlation patterns from training data rather than performing formal logical inference. CRAwDAD makes the implicit "internal dialogue" of human hypothesis testing explicit via Multi-Agent Debate (MAD), leveraging reasoning language models (RLMs) such as Qwen3 and DeepSeek-R1 as debate components.

Architecture

CRAwDAD uses a dual-agent design without a separate judge model:

• Agent A (Proposer): provides structured causal reasoning (e.g., extracting the causal graph, formalizing the query).
• Agent B (Critic): audits the reasoning, challenging logic and flagging fallacies.

Design highlights:

• Heterogeneity: different models (Qwen3 vs DeepSeek-R1) ensure diverse perspectives and avoid shared blind spots.
• Explicit confidence modeling: each agent attaches a 0.0–1.0 confidence score, enabling analysis of persuasion dynamics and correction of confidently wrong answers.
• Prompt engineering: an initial prompt enforces a 7-step causal reasoning pipeline (extract causal graph, determine query type, formalize the query, etc.) to prevent reliance on linguistic correlation.

Debate Protocol

1. Initial response: a randomly chosen agent gives an answer with confidence. 2. Critique phase: the other agent analyzes the answer for logical gaps or calculation errors. 3. Defend or revise: the first agent defends or updates its conclusion. 4. Early stopping: the debate ends immediately upon consensus. 5. Round cap: debates are limited to roughly 4 rounds if no agreement is reached.

Experimental Setup

Dataset: CLadder, a benchmark linking natural-language questions to formal causal models, covering all three rungs of Pearl's causal hierarchy — Rung 1 (seeing/association), Rung 2 (doing/intervention), and Rung 3 (imagining/counterfactuals, the hardest).

Models: Qwen3-32B and DeepSeek-R1-Distill-Qwen-32B.

Results

| Model | Task | Single-agent | Dual-agent debate | Gain | |---|---|---|---|---| | DeepSeek-R1 | Overall | 78.03% | 87.45% | +9.42% | | DeepSeek-R1 | Counterfactual | 67.94% | 80.04% | +12.10% | | Qwen3 | Overall | 84.16% | 89.41% | +5.25% | | Qwen3 | Counterfactual | 71.53% | 80.35% | +8.82% |

Key findings:

• The largest improvements occur on counterfactual reasoning, suggesting debate especially helps with complex "what-if" scenarios.
• DeepSeek-R1 is more easily persuaded, frequently revising initially wrong answers when challenged by Qwen3.
• Models rarely express mid-range confidence (65–80%), tending toward extreme confidence or uncertainty.

Conclusion and Future Work

CRAwDAD demonstrates that reasoning models are strong building blocks for multi-agent causal reasoning systems, correcting errors caused by conflating correlation with causation and selection biases (e.g., Collider Bias). Future work includes exploring more diverse model pairings, scaling debates to more complex causal graphs, and optimizing the trade-off between debate rounds and compute cost.