Latent Dynamics Reasoning (LDR): Extrapolative Video World Models via Explicit Kinematic Integration
Forum topic · 小凯 · 2026-08-11
Summary
This paper introduces Latent Dynamics Reasoning (LDR), a framework for video world models that explicitly captures underlying motion laws from pixels rather than merely fitting visual appearance. LDR formulates latent transitions as kinematic integration: lower-order dynamics (up to second order) are numerically integrated, while a model regresses only the third- and higher-order residuals that drive rollouts. The integration operates on a structured latent rather than dense convolutional features to improve extrapolation. Evaluated on the controlled PhyWorld benchmark across five tasks (uniform motion, projectile, collision, bounce, and approach), LDR significantly narrows the gap between in-distribution and out-of-distribution error by over 20x compared with a video diffusion baseline, while using 26x fewer parameters and running 143x faster at 256x256 resolution. Notably, LDR generalizes under severe distribution shift, e.g., trained only on left-to-right red balls, it correctly predicts right-to-left blue square motion. The authors claim this is the first video world model to extrapolate learned dynamics outside the training distribution.
Key points
- Problem: Leading video diffusion models fit pixels without explicitly modeling how pixels transit over time, so they produce visually plausible frames that may violate physical laws.
- Method: The authors propose Latent Dynamics Reasoning (LDR), which casts the latent transition as an explicit kinematic integration:
- Lower-order dynamics (up to second order) are numerically integrated.
- The model regresses only the third- and higher-order residual that drives the rollout.
- Integration runs on a structured latent instead of dense convolutional features, aiding extrapolation.
- Benchmark: Following PhyWorld, evaluation uses a controlled white-box physics benchmark with five tasks (uniform motion, projectile, collision, bounce, and approach), focusing on out-of-distribution (OOD) scenarios.
- Results at 256^2 resolution:
- The gap between in-distribution and OOD error is reduced by more than 20x compared with the video diffusion baseline.
- Uses 26x fewer parameters.
- Runs 143x faster.
- Improvements hold for both single-task and joint-task training.
- Generalization under severe shift: Trained only on left-to-right moving red balls, LDR correctly predicts right-to-left moving blue squares, demonstrating genuine OOD extrapolation of learned dynamics.
- Claim: To the authors' knowledge, LDR is the first video world model shown to extrapolate learned dynamics outside the training distribution.
Reference
- arXiv: 2508.03804 — https://arxiv.org/abs/2508.03804
- Project page: https://lat-dyn-reason.github.io/
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178633349