YoCausal: Do Video Generation Models Really Understand Causality, or Just Watch Time Flow?
> Core finding: 13 state-of-the-art video diffusion models were tested end to end. They can tell whether a video plays forward or backward—but this is not causal understanding, only statistical muscle memory for temporal direction. When it comes to genuinely causal judgments like "a broken egg cannot un-break," models broadly fail. The authors borrow a tool from infant cognitive science: show babies a physics-violating magic trick and they stare longer—a "violated expectation." YoCausal ports the same logic into evaluating video generation models.
---
The One-Sentence Takeaway
Video diffusion models (VDMs) are billed as "world models"—their outputs look coherent and physically plausible. But does that coherence come from truly understanding causality, or from statistical memory that "time usually flows this way"? YoCausal is the first benchmark to systematically answer this. Its zero-cost trick: play real videos backward. If a model truly understands causality, it should be more "surprised" by a physically impossible sequence like a shattered egg reassembling—not because it "looks weird," but because a causal relation was violated.
---
Why This Matters: A Litmus Test for World Models
"World model" is an overused term
Sora, Wan, HunyuanVideo—these models produce ever more realistic video, and the industry calls them "world models," implying they grasp physics, causality, and object permanence.
But a gap separates two things:
- Statistical coherence: the model learned "pixels typically change like this"
- Causal understanding: the model knows "this change causes that one, and not the reverse"
- Forward video: physically possible event sequence (egg drops and shatters)
- Reversed video: time-reversed, physically impossible (shattered egg springs back whole)
- Model behavior: if the model "understands" causality, it should be more "surprised" by the reversed video—manifesting as higher denoising loss
- Causal subset \(D_c\): irreversible causal events (breaking, burning, growing, collapsing)
- Non-causal subset \(D_{nc}\): motion without causal structure (walking, waving, drifting clouds)
- RSI = can you tell the video is reversed? (even someone who never cooks notices eggs being "un-scrambled")
- CCI = do you notice because you understand cooking, or just because "backward motion looks weird"?
- Mechanical (breaking, collapsing): relatively easy, because visual change is drastic
- Chemical (burning, dissolving): moderate
- Biological (growing, decaying): hardest—long time scales, gradual change
- Mask cause frames and predict effects
- Contrastive learning: penalize physically impossible sequences
- Causal-intervention data augmentation
- Language models: perceiving syntax (Level 1) vs. understanding semantics (Level 2)
- Robotics: perceiving object motion (Level 1) vs. understanding affordances (Level 2)
- Program synthesis: perceiving code structure (Level 1) vs. understanding execution causality (Level 2)
- Paper: YoCausal: How Far is Video Generation from World Model? A Causality Perspective
- arXiv: 2605.30346
- Authors: You-Zhe Xie, Yu-Hsuan Li, Jie-Ying Lee, Kaipeng Zhang, Yu-Lun Liu, Zhixiang Wang
- Institutions: National Yang Ming Chiao Tung University, Alaya Studio
- Project page: https://www.youzhexie.me/papers/YoCausal
- Code: https://github.com/youzhe0305/YoCausal
- Metrics: RSI (temporal arrow perception), CCI (causal cognition), RSI(Hd/Hnd) (human-distinguishable / non-distinguishable subsets)
Analogy: a model watching countless coffee-plus-milk videos learns the sequence "milk diffuses into coffee." But does it understand diffusion as an irreversible thermodynamic process? Or would reversed footage of milk gathering back into a droplet merely strike it as "a sequence I haven't seen," rather than "physically impossible"?
That is exactly what YoCausal measures.
---
Method: The Violation-of-Expectation Paradigm, Borrowed from Infant Psychology
Violation of Expectation (VoE) in cognitive science
Developmental psychologists show infants a ball rolling behind a screen; the screen lifts and the ball is gone—infants stare longer. Not because they are confused, but because their expectations about the world were violated. Balls do not vanish; this causal rule is already internalized.
YoCausal digitizes the same logic:
Why is it "zero cost"?
Traditional causal benchmarks suffer from expensive counterfactual samples: testing "what if X hadn't happened" requires manual synthesis, controlled recording, or 3D physics simulation—with an ever-present sim-to-real gap.
YoCausal's trick: temporal reversal is a natural counterfactual. No annotation, no synthesis, no simulation. Take a YouTube clip, reverse it, done. Any real-world video instantly becomes a causal test sample.
This means the dataset can scale without limit. General scenes, physical phenomena, human behavior, animal behavior—any video with causal structure works.
---
A Two-Level Evaluation Framework: Separating "Perceiving Time" from "Understanding Causality"
This is YoCausal's most elegant design. Many assume "can tell forward from backward" means understanding causality. It does not.
Level 1: Reverse Surprise Index (RSI) — do you perceive the arrow of time?
For forward video \(x_f\) and reversed video \(x_r\), add the same noise \(\epsilon\) and compute denoising losses separately:
RSI measures how often the model is "surprised" by reversed video. But there is a trap—some reversed videos do not violate physics; they just look odd. A person walking forward vs. backward has no causal issue, only a different time direction.
Level 2: Causality Cognition Index (CCI) — do you truly understand causality?
Key insight: not all videos contain visible causal relations.
YoCausal uses a vision-language model (VLM) to automatically classify videos:
Then compute RSI on each subset:
CCI is the real gold. If a model only has a "statistical time preference," its RSI should be similar on both subsets—CCI near 0. If it truly understands causality, it should show stronger "surprise" on the causal subset—CCI significantly above 0.
Analogy:
---
The Physical Exam Results for 13 SOTA Models
The paper tested 13 video diffusion models, covering open-source and commercial APIs. Key findings:
1. Temporal awareness ≠ causal understanding
Models vary widely on RSI, but CCI is generally low. Perceiving the arrow of time and understanding causality are two independent capabilities; current models mostly develop the former.
2. Scaling does help
Larger models and newer architectures show higher CCI. This points a direction for world-model development: bigger models plus better training objectives may gradually approach causal understanding.
3. Open-source models lag behind
Open-source VDMs fall significantly short of human-level causal cognition (and of some commercial APIs). This does not mean open-source is hopeless—it shows causal understanding does not emerge automatically; it needs targeted training signals.
4. Causal types differ
Models perform differently across causal event types:
---
Technical Detail: Why Can Denoising Loss Serve as "Surprise"?
A video diffusion model's training objective is denoising: learning to recover clean video from noise. If a sequence is physically "unreasonable," the model never saw that pixel-change pattern in training, so its denoising loss rises—it doesn't know how to map noise to "plausible" frames.
It's not a perfect proxy. Critics note that reversing complex scenes may raise loss simply because they are harder to denoise, not because causality was violated. But YoCausal's differenced CCI design partially controls for this confound—if "hard to denoise" were the main driver, the non-causal subset's RSI would be equally high.
---
Implications for the Field
For world-model research
YoCausal provides a low-cost, scalable, real-world-data-based causal evaluation tool. Researchers previously had to choose between synthetic bouncing balls (unrealistic) or expensive 3D simulation (not scalable). Now anyone can take a real video, reverse it, and run the test.
For video generation developers
If your model has high RSI but low CCI, you are overfitting statistical temporal patterns. Consider explicit causal training signals:
For other domains
The two-level framework (perception vs. understanding) generalizes:
---
Key Information
> "Telling forward from backward is not understanding physics. YoCausal sets a causal bar for 'world models.'"