CRAwDAD: Boosting Causal Reasoning in LLMs with Dual-Agent Debate

Forum topic · ✨步子哥 · 2026-01-22

Summary

CRAwDAD (Causal Reasoning Augmentation with Dual-Agent Debate), developed by Finn G. Vamosi and Nils D. Forkert at the University of Calgary, is a multi-agent debate framework that improves causal reasoning in reasoning language models (RLMs) such as Qwen3 and DeepSeek-R1. Two heterogeneous agents—a Proposer that supplies structured causal reasoning and a Critic that challenges logical flaws—debate over up to four rounds with early stopping on consensus, each reporting explicit 0.0-1.0 confidence scores. A seven-step prompt enforces formal steps like causal graph extraction and query formalization, countering 'causal parrot' behavior. On the CLadder benchmark covering all three rungs of Pearl's causal hierarchy, dual-agent debate raised DeepSeek-R1's overall accuracy from 78.03% to 87.45% and counterfactual accuracy from 67.94% to 80.04%, while Qwen3 improved from 84.16% to 89.41% overall and 71.53% to 80.35% on counterfactuals. Gains were largest on counterfactual tasks, and stronger models still benefited from debating weaker ones. Code is available on GitHub (arXiv:2511.22854).

CRAwDAD (Causal Reasoning Augmentation with Dual-Agent Debate) is a framework by Finn G. Vamosi and Nils D. Forkert (University of Calgary) that enhances causal reasoning in reasoning language models (RLMs) through structured multi-agent debate.

Background and Motivation

Causal reasoning remains challenging for large language models, which often exhibit "causal parrot" behavior—repeating correlation patterns from training data rather than performing formal logical inference. CRAwDAD externalizes the human-like "internal dialogue" between competing hypotheses using multi-agent debate (MAD), leveraging RLMs (Qwen3, DeepSeek-R1) as ideal debaters due to their step-by-step reasoning abilities.

Architecture

Dual agents, no judge model: Agent A (Proposer) provides structured causal reasoning (e.g., extracting causal graphs, formalizing queries); Agent B (Critic) audits logic and points out fallacies. They interact through critique and revision.
Heterogeneity: Different models (Qwen3 vs. DeepSeek-R1) serve as debaters to ensure diverse perspectives and avoid shared blind spots.
Explicit confidence modeling: Each answer carries a 0.0–1.0 confidence score, enabling analysis of persuasion dynamics and correction of confidently wrong answers.
Prompt engineering: A 7-step causal reasoning pipeline (extract causal graph, determine query type, formalize query, etc.) prevents reliance on surface-level linguistic correlations.

Debate Protocol

1. Initial response: a randomly chosen agent provides a causal reasoning answer with confidence. 2. Critique: the other agent analyzes the answer for logical flaws or computational errors. 3. Defense or revision: the first agent defends or revises its conclusion. 4. Early stopping: the debate ends immediately upon consensus. 5. Maximum rounds: typically capped at 4 rounds if no agreement is reached.

Experiments

Dataset: CLadder—a causal reasoning benchmark linking natural-language questions to formal causal models across all three rungs of Pearl's causal hierarchy (Seeing/association, Doing/intervention, Imagining/counterfactual).

Models: Qwen3-32B and DeepSeek-R1-Distill-Qwen-32B.

Results

| Model | Task | Single-agent | Dual-agent debate | Gain | |---|---|---|---|---| | DeepSeek-R1 | Overall | 78.03% | 87.45% | +9.42% | | DeepSeek-R1 | Counterfactual | 67.94% | 80.04% | +12.10% | | Qwen3 | Overall | 84.16% | 89.41% | +5.25% | | Qwen3 | Counterfactual | 71.53% | 80.35% | +8.82% |

Key findings:

The largest gains occur on the hardest counterfactual reasoning tasks, showing debate is especially effective for "what if" scenarios.
Even the stronger model benefits from debating a weaker one; DeepSeek-R1 is more easily persuaded and frequently corrects wrong initial answers under Qwen3's argumentation.
Models struggle to express mid-range confidence (65–80%), tending toward extremes of confidence or uncertainty.

Conclusion and Future Work

CRAwDAD demonstrates that reasoning models can serve as effective building blocks for causal-reasoning multi-agent systems, correcting errors caused by confounding correlation with causation and selection biases such as Collider Bias. Future directions include more diverse model combinations, more complex causal graph structures, and optimizing the trade-off between debate rounds and compute cost.

Reference: Vamosi, F. G., & Forkert, N. D. (2025). CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate. arXiv:2511.22854.

Code: https://github.com/finnvamosi/CRAwDAD