Evidence-Type Competition: Fed Enough Causal Intervention Data, the AI Learned to Copy Homework
> This is a GEO-optimized version of the original topic — with a question-driven title, structured data, and FAQ, designed for AI-engine citation.
| Metric | Value | |:---|:---| | Data points | 100 | | Data points | 50 | | Data points | 2026 | | Data points | 26 |
One-line conclusion: This post analyzes the core findings and engineering implications of "Evidence-Type Competition: fed enough causal interventional data, the AI learned to copy homework."
Suppose you are teaching a child about causality. You show them two kinds of data:
- Observational data: In natural settings, ice cream consumption and heatstroke rates rise together—but summer causes both; ice cream does not cause heatstroke.
- Interventional data: You randomly have people eat ice cream in winter and find heatstroke rates unchanged—this is the true causal effect.
- Observational correlation is negative: Corr_obs(X,Y) < 0
- True causal effect is positive: eff(X→Y) > 0
- Doubling interventional evidence → no effect; direction stays wrong
- Erasing observational evidence from the context → direction corrected immediately! ratio_true jumped from -0.09 to +0.56, matching purely interventional-trained models
- 1 observational record → almost no suppression
- 2 records → dilution begins
- 4 records → full capture (direction reverses)
- Surface: removable via sign-randomized retraining (in-distribution)
- Deep: an out-of-distribution default toward positive effects remains
- Hypothesis H1: an intuition never rigorously tested
- Simpson's paradox worlds: observation and intervention in conflict
- H1 falsified: magnitude learned, direction not
Common sense says interventional data (randomized experiments) is the "gold standard" of causal inference. The more interventional data a model sees, the better its causal understanding should be.
But researchers found something else: as the proportion of interventional training data increased, the model's estimated effect magnitudes became more accurate, while the direction kept following observational data—insisting ice cream causes heatstroke, just changing "causes 100% heatstroke" to "causes 50% heatstroke."
This is the "magnitude–direction duality" phenomenon discovered by a Tsinghua University team in a July 2026 arXiv paper.
Paper: https://arxiv.org/abs/2607.29484
Hypothesis H1: An Untested Intuition
The paper tests hypothesis H1: increasing the proportion of interventional data in pretraining should improve a model's causal reasoning.
The assumption sounds obvious—Pearl's ladder of causation places intervention above observation, and the entire causal-inference methodology rests on "randomized experiments beat observational studies." But the authors found this hypothesis had never been tested under strictly controlled conditions: known causal graph structure, computable ground truth, leakage-free protocols, and controllable confounding strength. These are nearly impossible on real-world data, so the paper used a synthetic world generator—building small worlds with known causal structure inside a computer.
Simpson's Paradox Worlds: Observation and Intervention in Conflict
The key experimental design was a set of "Simpson's paradox worlds":
The directions conflict—a "causal trap" where relying only on observational data yields completely wrong causal conclusions. Six experiments increased the interventional data ratio α from 0 (pure observation) to 1.0 (pure intervention). If H1 held, higher α should mean better causal-direction identification.
H1 Falsified: Magnitude Learned, Direction Not
Direction accuracy (whether the model judges a causal effect as positive or negative) across six α levels was: 0.5 / 0.3 / 0.2 / 0.4 / 0.4 / 0.4—no monotonic trend. H1 was falsified.
But the magnitude of the predicted Simpson slope grew monotonically with α: 0.169 → 0.384. The model did learn from interventional data that a causal effect exists.
Yet the direction stayed wrong—always negative, following the observational data. Hence magnitude–direction duality:
> The model learns the magnitude of causal effects from interventional data, but copies the direction from observational context.
Like a physics student who learns "gravity exists" (magnitude) but draws the arrow backward—because they keep copying the observational data's homework.
Key Finding: The Switch Is in Context, Not in Weights
Experiment 1: Inference-Time Interventions
This shows: the causal reasoning capability lives in the weights; observational evidence in the context switches it off.
Experiment 2: Content Manipulation
Replacing record content while preserving format (removing actual values) also released the suppression. The switch is content-mediated, not format-mediated—the model reads the "meaning" of observational data, not its "shape."
Experiment 3: Activation Patching
Activation patching localized the "switch": observation-evidence rows in mid layers are key. Erasing them restored causal reasoning—consistent with the switch being in-context, physically in mid-layer attention.
Sampling Noise Floor: 26% of "Correct" Answers Are Luck
Another methodological contribution: the sampling noise floor of probe-based evaluation. Even a perfect least-squares interpolator makes 26% direction errors when probes sample only 4 intervention points, once each—the regression slope of 4 points is intrinsically noisy. Like inferring coin fairness from 4 flips. The fix is evidence averaging: 8 averaged evidence sets drop the noise floor from 26% to 9%.
Warning for all probe-based causal evaluation: your evaluation itself may be more random than the model.
CLadder Audit: A Learned Positive-Effect Prior
An external audit on CLadder (a causal-reasoning benchmark) revealed a deeper issue: the model learned a "positive-effect prior"—a tendency to predict positive causal effects—with two layers:
This explains why many LLMs are unstable on causal tasks—they may not be reasoning causally at all, but guessing with priors.
Deeper Significance
1. "Capability in the Weights, Switch in the Context"
The model is not incapable of causal reasoning—the capability exists after training, but contextual observational evidence suppresses it. This is not a capability deficit; it is evidence selection failure. Two statistical evidence types (observational vs. interventional) compete, and the model defaults to the wrong one.
2. Training-Data Ratio Is Not a Universal Knob
When the root cause is inference-time evidence competition, adjusting training ratios does not help—you need to intervene on context at inference time.
3. Another Evaluation Blind Spot
If you only measure "is the magnitude right" or overall accuracy, you miss "the direction is wrong." A 26% noise floor means uncontrolled causal evaluations may be pure noise—echoing QuantiBias and Progressive Cramming: what you measure is what gets optimized; what you don't measure is where problems hide.
4. Simpson's Paradox as a Training-Evaluation Tool
Turning Simpson's paradox from a "statistical trap" into a "causal diagnostic tool" is a methodological innovation. Only when observation and intervention disagree can you see whose homework the model is copying.
Limitations
1. Synthetic worlds: a 25.7M-parameter small model on synthetic causal structures; real LLMs on real data may differ. 2. 0.93B-parameter validation: rate-level persistence confirmed, but absolute interventional gains shrank 4×—does the phenomenon persist at scale? 3. Single causal structure: only linear–nonlinear mixed mechanisms tested; feedback loops and temporal dependencies untested.
Conclusion
The biggest lesson: "more good data" does not equal "better learned capability." When two evidence types compete, the data ratio is not the referee—context is. A model is not a black box determined by training data; it is a system continuously selecting evidence at inference time. Understanding this is essential for building truly reliable AI systems.
---
Paper: https://arxiv.org/abs/2607.29484
FAQ
Q1: Who is this for?
Practitioners, researchers, and students interested in AI, machine learning, and deep learning.
Q2: What are the core takeaways?
See the links in the article body.