Why LLMs Fail at Causal Discovery and How Interventional Agents (A-CBO) Fix It

Forum topic · 小凯 · 2026-05-29

Summary

This paper (arXiv:2605.27567) by Amartya Roy and Sonali Parbhoo explains why large language models fail at causal discovery. The authors prove the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data. Any attempt to do so would require unbounded growth of internal representations, violating the conditions under which these training methods work. This is formalized as a kernel obstruction theorem, showing the limitation is intrinsic to the learning paradigm itself rather than a specific model or dataset. To address it, they propose Agentic Causal Bayesian Optimization (A-CBO): a frozen LLM acts as an interventional oracle answering targeted queries, while an external Bayesian optimization loop concentrates belief over candidate graphs in log rounds. Because decisions operate outside the space where the obstruction applies, A-CBO provably converges while the underlying model remains unchanged. On a new 24-variable Extended Corr2Cause benchmark with 18K test samples, A-CBO significantly outperforms fine-tuning and preference-optimization baselines.

Paper Overview

Field: AI
Authors: Amartya Roy, Sonali Parbhoo
Published: 2026-05-28
arXiv: 2605.27567

Key Points

Causal discovery is a cornerstone of scientific reasoning, yet LLMs perform it unreliably. Benchmarks show even fine-tuned models plateau on simple causal graphs and degrade as complexity grows — with the root cause previously unexplained.
Fundamental failure: Supervised fine-tuning, direct preference optimization, and in-context learning all yield predictors that cannot distinguish causal graphs generating similar observational data.
Distinguishing such graphs would require unbounded growth of the model's internal representations, violating the very conditions under which these training methods operate.
This is formalized as a kernel obstruction theorem: the limitation is intrinsic to the learning paradigm, not any particular model or dataset.
Proposed solution — Agentic Causal Bayesian Optimization (A-CBO):
A frozen language model serves as an interventional oracle, answering targeted queries.
An external Bayesian optimization loop concentrates belief over candidate causal graphs in log rounds.
Because decisions operate outside the space where the kernel obstruction applies, A-CBO provably converges while the underlying model stays unchanged.
Results: On the new 24-variable Extended Corr2Cause benchmark (18K test samples), A-CBO significantly outperforms fine-tuning and preference-optimization baselines.

Abstract (Original)

Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine-tuned models plateau on simple causal graphs and degrade as complexity grows, but why they fail has not been established. We prove the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data, and any attempt to do so requires the model's internal representations to grow unboundedly, violating the very conditions under which these methods work. We formalize this as a kernel obstruction theorem, establishing that the limitation is intrinsic to the learning paradigm, not any particular model or dataset. We propose Agentic Causal Bayesian Optimization (A-CBO), w...

--- *Auto-collected on 2026-05-29*