NeuROK: Generative 4D Neural Object Kinematics

Forum topic · 小凯 · 2026-06-01

Summary

NeuROK (Neural Object Kinematics) is a paper by Chen Geng, Guangzhao He, Yue Gao, Yunzhi Zhang, Shangzhe Wu, and Jiajun Wu, posted on arXiv (2605.30347) on May 28, 2026, in the computer vision field. The work addresses generative simulated 4D dynamics: realistic temporal deformations of static objects under diverse physical conditions. Existing methods typically assume predefined physical models and estimate parameters via system identification, restricting them to specific object categories and small datasets. The authors instead propose learning a data-driven kinematic state parameterization of object-centric physical systems. They learn a latent space representing all possible states of an object, together with a decoder mapping arbitrary latent samples to plausible deformed shapes. A Transformer-based encoder-decoder model is trained on a curated large-scale 4D dataset. From the Lagrangian mechanics perspective, dynamics only need to be modeled within this low-dimensional latent space, which greatly simplifies generating simulated dynamics. Experiments across diverse dynamic object types demonstrate the effectiveness and generality of the framework, significantly outperforming prior work.

Paper Overview

Research Area: Computer Vision Authors: Chen Geng, Guangzhao He, Yue Gao, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu Published: 2026-05-28 arXiv: 2605.30347

Abstract

Data-driven approaches have revolutionized 3D vision, enabling Transformers to effectively reconstruct and generate static 3D objects. However, generating *simulated* 4D dynamics — realistic temporal deformations of static objects under various physical conditions — remains challenging and often ad hoc, despite being crucial for building comprehensive 3D world models.

Most existing methods assume predefined physical models and use system identification to estimate their parameters, which limits these approaches to specific categories and small-scale datasets. The authors argue that these limitations can be overcome by learning a data-driven kinematic state parameterization of object-centric physical systems.

Approach

Specifically, the method learns:

A latent space representing all possible states of an object.
A decoder that maps arbitrary samples from this latent space to plausible deformed shapes of the object.

This parameterization is termed Neural Object Kinematics (NeuROK), learned with a Transformer-based encoder-decoder model on a curated large-scale 4D dataset.

This formulation significantly simplifies generating simulated dynamics: from the Lagrangian mechanics perspective of classical physics, dynamics only need to be considered within the low-dimensional latent space.

Results

The authors demonstrate the effectiveness and generality of this neural simulation framework across a variety of dynamic object types, markedly outperforming prior work.

--- *Auto-collected on 2026-06-01*