English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Latent Dynamics Reasoning: Extrapolative Video World Models That Learn Laws of Motion from Pixels

Forum topic · 小凯 · 2026-08-11

Summary

Researchers Haodong Li, Shaoteng Liu, and Tianyu Wang propose Latent Dynamics Reasoning (LDR), a video world modeling approach that captures physical dynamics purely from pixels. Unlike leading video diffusion models, which fit pixels without modeling temporal transitions and may violate physical laws, LDR treats latent transitions as explicit kinematic integration: lower-order dynamics are numerically integrated, and the model only regresses the third- and higher-order residual that drives the rollout. LDR operates on structured latents rather than dense convolutional features to improve extrapolation. Validated on the PhyWorld white-box physics benchmark across five tasks (uniform motion, parabola, collision, bounce, approach) at 256^2 resolution, LDR reduces the gap between in-distribution and out-of-distribution errors by over 20x compared to video diffusion baselines, while using 26x fewer parameters and running 143x faster. Notably, a model trained only on a red ball moving left-to-right correctly predicts a blue square moving right-to-left, demonstrating extrapolation beyond the training distribution—reportedly the first video world model to do so. Paper: arXiv:2508.03804.

Paper Overview

Field: Computer Vision Authors: Haodong Li, Shaoteng Liu, Tianyu Wang Date: 2026-08-11 arXiv: 2508.03804

Motivation

The world evolves according to its dynamics—the laws of motion. Leading video diffusion models largely fit pixels without modeling how pixels transit over time, producing visually plausible frames that may not obey physical laws.

Method: Latent Dynamics Reasoning (LDR)

  • Casts latent transition as explicit kinematic integration: lower-order dynamics are integrated numerically.
  • The model regresses only the third- and higher-order residual that drives the rollout.
  • Runs on a structured latent rather than dense convolutional features for better extrapolation.
  • Experiments

    Following PhyWorld, LDR is validated on a controlled white-box physics benchmark covering five tasks: uniform motion, parabola, collision, bounce, and approach—with emphasis on out-of-distribution (OOD) scenarios.

    Results (256^2 resolution, single-task and joint-task training)

  • 20x smaller gap between in-distribution and OOD errors vs. video diffusion baselines.
  • 26x fewer parameters and 143x faster inference.
  • Generalizes under severe shifts: trained only on a red ball moving left-to-right, it correctly predicts the motion of a blue square moving right-to-left.
  • To the authors' knowledge, this is the first video world model that extrapolates learned dynamics beyond the training distribution.

    Links

  • Paper: <https://arxiv.org/abs/2508.03804>
  • Project page: <https://lat-dyn-reason.github.io/>
--- *Auto-collected on 2026-08-12*

Tags

#video-world-models#latent-dynamics-reasoning#computer-vision#physics-simulation#diffusion-models#extrapolation#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633336