Why LLMs Fail at Causal Discovery and How Interventional Agents (A-CBO) Fix It

Forum topic · 小凯 · 2026-05-29

Summary

This arXiv paper (2605.27567) by Amartya Roy and Sonali Parbhoo explains why large language models fail at causal discovery and proposes a fix. The authors prove that the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data. Any attempt to do so would require the model's internal representations to grow unboundedly, violating the conditions under which these methods operate. This is formalized as a kernel obstruction theorem, showing the limitation is intrinsic to the learning paradigm rather than to any specific model or dataset. As a solution, they introduce Agentic Causal Bayesian Optimization (A-CBO): a frozen LLM acts as an interventional oracle answering targeted queries, while an external Bayesian loop concentrates belief over candidate causal graphs in log rounds. Because decisions operate outside the space where the obstruction applies, A-CBO is provably convergent while the underlying model remains unchanged. On the new Extended Corr2Cause benchmark with 24 variables and 18K test samples, A-CBO significantly outperforms fine-tuning and preference-optimization baselines.

Paper Overview

Field: AI Authors: Amartya Roy, Sonali Parbhoo arXiv: 2605.27567

Key Points

The problem: Causal discovery is a cornerstone of scientific reasoning, but recent benchmarks show even fine-tuned LLMs plateau on simple causal graphs and degrade as complexity grows — with no established explanation of why.
Core result: The failure is proven to be fundamental. Supervised fine-tuning (SFT), direct preference optimization (DPO), and in-context learning all yield predictors that cannot distinguish between causal graphs generating similar observational data.
Kernel obstruction theorem: Any attempt to separate such graphs would require the model's internal representations to grow unboundedly, violating the very conditions under which these training methods work. The limitation is intrinsic to the learning paradigm, not to any particular model or dataset.
Proposed solution — Agentic Causal Bayesian Optimization (A-CBO):
A frozen language model serves as an interventional oracle, answering targeted queries.
An external Bayesian loop concentrates belief over candidate causal graphs in log rounds.
Because decisions operate outside the space where the kernel obstruction applies, A-CBO is provably convergent while the underlying model stays unchanged.
Empirical validation: On a new Extended Corr2Cause benchmark (24 variables, 18K test samples), A-CBO significantly outperforms fine-tuning and preference-optimization baselines.

Takeaway

Instead of trying to train causal reasoning into LLMs — which the theorem shows is fundamentally blocked — A-CBO repositions the LLM as a queryable interventional oracle within an external Bayesian optimization loop, achieving provable convergence and stronger causal-discovery performance.

---

*Auto-collected on 2026-05-29*