This forum post reviews the arXiv paper "Reasoning Models Are Accurate but Unsound on Identification" (arXiv:2610.03519, October 2, 2026) by Arman Behnam and Binghui Wang of Illinois Institute of Technology. The author frames it as part one of a "gatekeeper question" trilogy about whether we can trust the judgments of causal AI systems — here, the specific question of whether reasoning models correctly decide when a causal query can be answered from observational data at all.
Key points
- The task: causal identification. Given a causal graph and a query P(y|do(x)), decide whether a formula using only observed variables exists (identifiable) or provably does not (Shpitser & Pearl, 2006). Errors split into *misses* (rejecting an answerable query, costly only in efficiency) and *false claims* (issuing a formula for an unidentifiable query — an error no observational data can ever expose, destroying soundness).
- Why existing benchmarks miss this: CLADDER/CaLM-style datasets are built with answers known in advance, so every question is answerable and refusal is never tested; human-labeled datasets are unreliable; string-matching scoring conflates form with semantics.
- CertID: labels from proofs, not people. The id algorithm on ADMGs is sound and complete: every estimand it returns is correct, every fail comes with a hedge-based non-identifiability proof. Two independent implementations agreed across ~14.9 million backend calls with zero divergence. Grading uses numerical evaluation on randomly sampled SCMs (k=2, tolerance 10⁻⁹), with a Schwartz-Zippel probabilistic polynomial identity guarantee (Theorem 2) that any pair of unequal formulas is almost surely caught.
- The benchmark: 1,200 certified instances (4–50 vertices): a 600-instance main pool of random and published graph families (asia, child, insurance, alarm, hepar2 projections), plus a 600-instance scale grid at 10–50 vertices. Random instances were generated (Aug 17, 2026) nearly four months after gpt-5.5's training snapshot, ruling out data contamination.
- Structural leakage: projected latent confounders form cliques, letting a name-blind classifier separate real from fake bidirected edges at AUC 0.825; making fake edges cliques dropped leakage to 0.652.
- Repair task + Theorem 1 (hedge destruction): an edge-deletion set restores identifiability iff it destroys all hedges — a min-cut problem. Models produced 92.6–97.0% valid, 96.8–99.3% minimal repairs unaided; flash with an id tool achieved 100% on both.
- A widely used causal-inference library returned numerically wrong estimands on 5 graphs, though its identifiability verdicts were correct — string matching can't distinguish "same semantics, different form"; numerical verification can.
- Behnam, A., & Wang, B. (2026). Reasoning Models Are Accurate but Unsound on Identification. arXiv:2610.03519.
- Shpitser, I., & Pearl, J. (2006). Identification of Joint Interventional Distributions in Recursive Semi-Markovian Causal Models. AAAI.
- Huang, Y., & Valtorta, M. (2006). Pearl's Calculus of Intervention Is Complete. UAI.
- Schwartz, J. T. (1980); Zippel, R. (1979). Probabilistic polynomial identity testing.
- Jin, Z., et al. (2023). CLADDER: Assessing Causal Reasoning in Language Models.
Findings
1. 97–100% decision accuracy for gemini-3.7-flash, gemini-3.1-pro-preview, and gpt-5.5, with answer rates tracking true identifiability rates — genuine discrimination, above the trivial "answer everything" baseline. Errors cluster at intermediate graph density (~10 bidirected edges); flash's false-claim rate peaks at 0.154 there and drops to 0 beyond 14 edges, where it simply refuses everything.
2. Explanations are withheld by default, not impossible. By default, correct refusals are bare 25-character verdicts. When prompted for structural reasons, 100% of 883 correct refusals named a concrete graph obstacle (~437–502 characters). Explainability is an interface property — but for flash, demanding reasons lowered correct-refusal rate from 97.0% to 95.3% and doubled false claims (9 → 14), a suggestive trade-off between transparency and stability.
3. Accuracy is a poor proxy for soundness. Accuracy varies by ~10 points across the three models, but false-claim rates differ 17× (flash vs. pro: 2× at 4-point accuracy gap; pro vs. gpt-5.5: 7× at 5-point gap). Additional evidence: estimand correctness of 98.8–98.9% (partially sampled), 4.5% (flash) / 2.5% (pro) inconsistency at temperature 0 with majority voting offering no gain, and scale effects that don't transfer across graph families. Even gpt-5.5 fabricated answers on 3 of 300 provably unanswerable queries — while the certified pipeline's false-claim rate is zero by construction.
Bonus results
Verdict
The post's closing metaphor: the access-control screen reads "zero anomalies today" — but the screen is wired to the gate itself. When gatekeepers are evaluated by their own readings, the readings are the failure. Upcoming sequels will cover external manipulation (JIL) and rebuilding an unbribable gate (NeutronGym).