YoCausal: Do Video Generation Models Understand Causality or Just 'Watch Time'?
> Core finding: 13 SOTA video diffusion models were evaluated—they can tell whether a video plays forward or backward, but this is not causal understanding, only statistical muscle memory for temporal patterns. When it comes to genuine causal judgments like "a broken egg cannot be unbroken," models generally fail. The authors borrow a tool from infant cognitive science: babies stare longer at physically impossible magic tricks—"violation of expectation." YoCausal brings the same logic into evaluating video generation models.
---
One-Sentence Summary
Video diffusion models (VDMs) are called "world models"—their generated videos look coherent and physically plausible. But does this coherence come from true causal understanding, or from statistical memory of "time usually flows this way"? YoCausal is the first benchmark to systematically answer this. It uses a zero-cost trick: playing real videos in reverse. If a model truly understands causality, it should be more "surprised" by a shattered egg reassembling—not because it "looks weird," but because the causal chain is violated.
---
Why This Matters: A Litmus Test for World Models
"World Model" Is an Overused Term
Sora, Wan, HunyuanVideo—these models generate increasingly realistic video, and the industry calls them "world models," implying they understand physics, causality, and object permanence.
But there is a gap between:
- Statistical coherence: the model learned "pixels usually change this way"
- Causal understanding: the model knows "this change causes that one, not vice versa"
- Forward videos: physically possible event sequences (an egg falls and breaks)
- Reversed videos: time-reversed, physically impossible (the broken egg bounces back whole)
- Model behavior: if the model "understands" causality, it should be more "surprised" by reversed videos—shown as higher denoising loss
- Causal subset \(D_c\): irreversible causal events (breaking, burning, growing, collapsing)
- Non-causal subset \(D_{nc}\): motion without causal structure (walking, waving, drifting clouds)
- RSI = can you tell the video is reversed? (even someone who can't cook notices the egg "un-frying")
- CCI = do you notice because you understand cooking, or just because "backward motion looks weird"?
- Mechanical (shattering, collapsing): relatively easy—dramatic visual changes
- Chemical (burning, dissolving): moderate
- Biological (growth, decay): hardest—long timescales and gradual changes
- Mask cause frames and predict effects
- Contrastive learning: penalize physically impossible sequences
- Causal-intervention data augmentation
- Language models: perceiving grammar (Level 1) vs. understanding semantics (Level 2)
- Robotics: perceiving object motion (Level 1) vs. understanding affordance (Level 2)
- Program synthesis: perceiving code structure (Level 1) vs. understanding execution causality (Level 2)
- Paper: YoCausal: How Far is Video Generation from World Model? A Causality Perspective
- arXiv: 2605.30346
- Authors: You-Zhe Xie, Yu-Hsuan Li, Jie-Ying Lee, Kaipeng Zhang, Yu-Lun Liu, Zhixiang Wang
- Institutions: National Yang Ming Chiao Tung University, Alaya Studio
- Project page: https://www.youzhexie.me/papers/YoCausal
- Code: https://github.com/youzhe0305/YoCausal
- Metrics: RSI (arrow-of-time perception), CCI (causal cognition), RSI(Hd/Hnd) (human-distinguishable / non-distinguishable subsets)
An analogy: a model watches countless videos of milk poured into coffee and learns the sequence "milk diffuses into coffee." But does it understand that diffusion is an irreversible thermodynamic process? Or would it only think "I've never seen this sequence" when shown milk un-diffusing—not "this is physically impossible"?
This is exactly what YoCausal tests.
---
Method: A "Violation of Expectation" Paradigm Borrowed from Infant Psychology
Violation of Expectation (VoE) from Cognitive Science
Developmental psychologists show babies a ball rolling behind a screen; the screen lifts and the ball is gone—babies stare longer. Not because they are "confused," but because their expectations about the world were violated. Balls don't vanish—a causal rule infants have internalized.
YoCausal digitizes the same logic:
Why "Zero Cost"?
Traditional causal benchmarks are expensive: testing "what if X hadn't happened" requires manual synthesis, controlled recording, or 3D physics simulation—with an ever-present sim-to-real gap.
YoCausal's trick: time reversal is a natural counterfactual. No annotation, no synthesis, no simulation. Take any YouTube video, reverse it—done. Any real-world video instantly becomes a causal test sample.
This means the dataset can scale without limit: general scenes, physical phenomena, human behavior, animal behavior—any video with causal structure works.
---
A Two-Level Framework: Separating "Perceiving Time" from "Understanding Causality"
This is YoCausal's most elegant design. Many assume "can tell forward from backward" means understanding causality. It does not.
Level 1: Reverse Surprise Index (RSI) — Do You Perceive the Arrow of Time?
For forward video \(x_f\) and reversed video \(x_r\), with the same noise \(\epsilon\), compute the denoising losses:
RSI measures how often the model is "surprised" by reversed videos. But there's a trap—some reversed videos don't violate physics, they just look "a bit odd." E.g., a person walking forward vs. backward. That's not a causality issue, just time direction.
Level 2: Causality Cognition Index (CCI) — Do You Actually Understand Causality?
Key insight: not all videos contain visible causal relationships.
YoCausal uses a vision-language model (VLM) to automatically classify videos:
Then compute RSI on each subset:
CCI is the real gold. If a model only has a "statistical temporal preference," its RSI should be similar on both subsets—CCI near 0. If it truly understands causality, surprise should be stronger on the causal subset—CCI significantly greater than 0.
Analogy:
---
Health Report on 13 SOTA Models
The paper evaluated 13 video diffusion models, covering open-source and commercial APIs. Key findings:
1. Time Perception ≠ Causal Understanding
RSI performance varies across models, but CCI is generally low. Perceiving the arrow of time and understanding causality are independent capabilities—current models mainly develop the former.
2. Scaling Does Help
Larger models and newer architectures show higher CCI. This points a direction for world models: larger scale + better training objectives may gradually approach causal understanding.
3. Open-Source Models Fall Behind
Open-source VDMs significantly lag human-level causal cognition (and some commercial APIs). This doesn't mean open-source is hopeless—it shows causal understanding doesn't emerge automatically and needs targeted training signals.
4. Causal Type Differences
Models perform differently across causal event types:
---
Technical Detail: Why Denoising Loss Works as a "Surprise Meter"
A video diffusion model's training objective is denoising: restoring clean video from noise. If a sequence is physically "implausible"—a pixel-change pattern never seen in training data—its denoising loss rises, because the model doesn't know how to map noise to "plausible" frames.
It's not a perfect proxy. Critics note that reversed complex scenes may have higher loss simply because they're harder to denoise, not because causality is violated. But YoCausal's differenced CCI design partially controls for this confound—if "hard to denoise" were the main factor, RSI on the non-causal subset would be equally high.
---
Impact on the Field
For World Model Research
YoCausal offers a low-cost, scalable, real-world-data-based causal evaluation tool. Researchers previously had to choose between synthetic bouncing balls (unrealistic) or expensive 3D simulation (not scalable). Now anyone can take a real video, reverse it, and run the test.
For Video Generation Developers
If your model has high RSI but low CCI, it is overfitting statistical temporal patterns. Consider explicit causal training signals:
For Other Fields
The two-level framework (perception vs. understanding) generalizes:
---
Key Information
> "Telling forward from backward is not understanding physics. YoCausal sets a causality bar for 'world models.'"