Evidence-Type Competition: Why More Causal Intervention Data Doesn't Fix LLM Direction Errors

Overview

A July 2026 arXiv paper from Tsinghua University reports a counterintuitive finding about how large language models handle competing causal evidence. The authors introduce magnitude–direction duality: when an LLM is trained on both observational and interventional data, it learns the correct *magnitude* of a causal effect but copies the *direction* from whichever evidence type dominates the prompt context.

> One-line conclusion: The paper falsifies H1 and reframes 'give the model more intervention data' as a necessary-but-insufficient fix. Real causal direction is gated by context, not by data ratios.

Source paper: https://arxiv.org/abs/2607.29484

The Hypothesis Being Tested (H1)

H1: Increasing the proportion of intervention data during pretraining should improve a model's causal reasoning ability.

This intuition is widespread—Pearl's Causal Hierarchy places intervention above observation, and the field treats randomized experiments as the gold standard. The authors note, however, that this assumption had never been tested under strict controls:

• Known causal graph structure
• Computable causal ground truth
• Leakage-free train/test splits
• Controllable confounder strength

The team therefore built a synthetic-world generator with a small (25.7M-parameter) model to test the hypothesis rigorously.

Simpson-Paradox Worlds

The experiment uses a deliberately adversarial setup:

• Observational correlation: Corr_obs(X,Y) < 0 (negative)
• True causal effect: eff(X→Y) > 0 (positive)

The two signals point in opposite directions—a causal trap. Any model that over-relies on observational data will infer the wrong sign. The authors sweep the intervention ratio α from 0 (pure observation) to 1.0 (pure intervention).

Key Result: H1 Is Falsified

Direction accuracy across six α levels: 0.5 / 0.3 / 0.2 / 0.4 / 0.4 / 0.4 — no monotonic improvement.

Yet Simpson-slope magnitude grows monotonically with α: 0.169 → 0.384. The model correctly learns *that* there is a causal effect, but the sign stays negative, tracking the observational data.

This is the magnitude–direction duality:

> The model learns effect magnitude from intervention data, but copies direction from observational context.

The Switch Is in Context, Not Weights

A series of inference-time interventions localize the failure:

Experiment 1 — Inference-Time Intervention

• Doubling intervention evidence: no effect; direction stays wrong.
• Erasing observational evidence: direction flips immediately (ratio_true jumps from -0.09 to +0.56, matching a pure-intervention-trained model).

The causal ability is already in the weights; observational context suppresses it.

Experiment 2 — Content Manipulation

• 1 observational record: minimal suppression
• 2 records: starts diluting
• 4 records: full capture (direction reverses)

Replacing the *content* (keeping the format) also releases suppression. The switch is content-mediated, not format-mediated.

Experiment 3 — Activation Patching

Erasing the observational rows in middle-layer activations restores causal reasoning. The suppression mechanism sits in middle-layer attention.

Sampling Noise Floor: 26% 'Correct' Is Luck

The paper introduces a methodological contribution: probe-based causal evaluation has a 26% sampling noise floor. Even a perfect least-squares interpolator will misjudge direction 26% of the time when probes sample only 4 intervention points, each measured once. A 4-point regression slope has too much variance.

Evidence averaging (8 sets) drops the noise floor from 26% to 9%. Implication: any causal evaluation that does not control sampling noise may itself be noisier than the model.

CLadder Audit: A Learned Positive-Effect Prior

External auditing on CLadder shows the model has acquired a positive-effect prior:

• Surface layer: removable through sign-randomized retraining (in-distribution).
• Deep layer: a default positive bias persists out-of-distribution.

LLMs may not be reasoning causally at all—they may be guessing from a prior.

Key Takeaways

1. Capability lives in weights; the switch lives in context. Two statistical evidence types (observation vs. intervention) compete, and the model defaults to the wrong one.

2. Training-data ratio is not a universal knob. When the root cause is inference-time evidence competition, you must intervene at inference time on context—not on the training mixture.

3. Evaluation blind spots multiply. Magnitude-only or overall-accuracy metrics hide direction errors. A 26% noise floor means uncontrolled causal evaluation can be pure noise.

4. Simpson's paradox as a diagnostic tool. When observation and intervention disagree in sign, a model's evidence-selection strategy is fully exposed—impossible to detect in sign-aligned worlds.

Limitations

1. Synthetic worlds: 25.7M parameters on synthetic causal structures; real LLMs on real data may behave differently. 2. Scale check only at 0.93B: absolute intervention gains shrink by 4×; whether the phenomenon persists in larger models is open. 3. Single causal family: only linear/nonlinear mixtures tested; feedback loops and temporal dependencies remain untested.

Conclusion

> "More good data" ≠ "better learned capability." When two evidence types compete, data ratio is not the judge—context is.

The LLM is not a black box decided by data; it is a system that performs continuous evidence selection at inference. Understanding this is a prerequisite for building reliably causal AI systems.

---

FAQ

Q1: Who is this for? AI, machine learning, and deep learning practitioners, researchers, and students.

Q2: Core takeaways?

• H1: an intuition never strictly tested
• Simpson-paradox worlds where observation and intervention oppose each other
• H1 falsified: magnitude learned, direction not