English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Learning How the World Evolves: Extrapolative Video World Models via Latent Dynamics Reasoning

Forum topic · 小凯 · 2026-08-11

Summary

Latent Dynamics Reasoning (LDR) is a new approach for video world models that captures physical dynamics purely from pixels, proposed by Haodong Li, Shaoteng Liu, and Tianyu Wang (arXiv:2508.03804). Unlike leading video diffusion models that fit pixels without modeling temporal transitions, LDR casts latent transitions as explicit kinematic integration: lower-order dynamics are integrated numerically while the model only regresses the third- and higher-order residual driving the rollout, operating on structured latents rather than dense convolutional features. Evaluated on a white-box physics benchmark (following PhyWorld) covering five tasks—uniform motion, parabola, collision, bounce, and approach—LDR shows strong out-of-distribution extrapolation: the gap between in-distribution and out-of-distribution errors is over 20x smaller than a video diffusion baseline at 256^2 resolution, while using 26x fewer parameters and running 143x faster. Notably, LDR generalizes under severe distribution shift, correctly predicting motion of unseen objects (e.g., a blue square moving right-to-left after training only on left-to-right red balls), reportedly the first video world model to extrapolate learned dynamics beyond its training distribution. Project page: https://lat-dyn-reason.github.io/

Paper Overview

Research Area: Computer Vision (CV) Authors: Haodong Li, Shaoteng Liu, Tianyu Wang Posted: 2026-08-11 arXiv: 2508.03804

Summary

The world evolves following its dynamics — its laws of motion. However, leading video diffusion models largely fit pixels without modeling how pixels transit over time. As a result, they generate visually plausible frames but may not accurately obey physical laws.

To capture dynamics purely from pixels, the authors introduce Latent Dynamics Reasoning (LDR):

  • LDR casts latent transitions as an explicit kinematic integration: lower-order dynamics are integrated numerically, and the model regresses only the third- and higher-order residual that drives the rollout.
  • To make this integration extrapolate better, LDR runs on a structured latent rather than dense convolutional features.
  • Following PhyWorld, LDR is validated on a controlled white-box physics benchmark spanning five tasks (uniform motion, parabola, collision, bounce, approach), focusing on out-of-distribution scenarios to test whether the model truly learned the underlying dynamics.

    Key Results

  • At 256^2 resolution, under both single-task and joint-task training, LDR's gap between in-distribution and out-of-distribution error is over 20x smaller than a video diffusion baseline.
  • LDR uses 26x fewer parameters and runs 143x faster.
  • LDR generalizes under severe distribution shift: trained only on a red ball moving left-to-right, it correctly predicts the motion of a blue square moving right-to-left.
To the authors' knowledge, this is the first video world model to extrapolate learned dynamics beyond its training distribution.

Project page: https://lat-dyn-reason.github.io/

--- *Auto-collected on 2026-08-12*

Tags

#video-world-models#latent-dynamics-reasoning#computer-vision#physics-simulation#out-of-distribution-generalization#arxiv#diffusion-models

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633349