ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning in VLA Models

Forum topic · 小凯 · 2026-10-28

Summary

ThinkWorld is a Vision-Language-Action (VLA) learning framework for robotic manipulation that grounds action reasoning in world dynamics. It represents each action thought as a Chain-of-Dynamics (CoD): a sequence of discrete motion tokens capturing intermediate dynamics between the current world state and a future state, serving as a common action representation across embodiments. The framework is first initialized with action-labeled robot demonstrations, then refined via reinforcement learning: the model predicts the future state induced by each CoD and reinforces those whose predictions better match observed outcomes. This lets unannotated videos—including human footage—supply direct supervision from observed state transitions, so human and robot experience refine the same reasoning space, addressing the trade-off between scalable data and explicit action grounding. Experiments in simulation and on real robots demonstrate effective human-to-robot transfer and strong generalization to unseen tasks and environments. The paper (arXiv 2610.37666) is authored by Chaehyun Kim, Chi-Pin Huang, and colleagues.

Overview

Research area: Robotics / VLA Authors: Chaehyun Kim, Chi-Pin Huang, Jisoo Kim, Min-Hung Chen, Yu-Chiang Frank Wang, Seungryong Kim, Fu-En Yang Published: 2026-10-27 arXiv: 2610.37666

Motivation

Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos offer much broader behavioral and environmental experience, yet learning from them involves a fundamental trade-off between scalable data and explicit action grounding.

Approach

ThinkWorld grounds action reasoning in world dynamics and strengthens it through reinforcement learning with world imagination:

Chain-of-Dynamics (CoD): each action thought is represented as a sequence of discrete motion tokens capturing the intermediate dynamics connecting the current world to a future state, serving as a common action representation across embodiments.
Initialization: CoD is first initialized with action-labeled demonstrations.
Reinforcement with world imagination: the model predicts the future induced by each CoD and reinforces those whose predictions better match the observed outcome.

This enables videos without action annotations to provide direct supervision from observed state transitions, allowing human and robot experience to refine the same reasoning space.

Results

Experiments in simulation and on real robots demonstrate effective human-to-robot transfer and strong generalization to unseen tasks and environments.

Abstract (from the paper)

> Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos provide much broader behavioral and environmental experience, while learning from them faces a fundamental trade-off between scalable data and explicit action grounding. We introduce ThinkWorld, a VLA learning framework that grounds action reasoning in world dynamics and strengthens it through reinforcement learning with world imagination. ThinkWorld represents each action thought as a Chain-of-Dynamics (CoD), a sequence of discrete motion tokens that captures the intermediate dynamics connecting the current world to a future state and serves as a common action representation across embodiments. We first initialize CoD with action-labeled demonstrations, then predict the future induced by each CoD and reinforce those whose predictions better match the observed outcome. This enables videos without action annotations to provide direct supervision from observed state transitions, allowing human and robot experience to refine the same reasoning space. Experiments in simulation and on real robots demonstrate effective human-to-robot transfer and strong generalization to unseen tasks and environments.