ReWAM: Rethinking Representations for World-Action Modeling in Robot Learning

Forum topic · 小凯 · 2026-10-01

Summary

ReWAM is a representation-centric world-action model that jointly learns robot policies and predicts future observations, treating the representation space as an interface between control and prediction. Through controlled comparisons, the authors show that neither reconstruction fidelity nor pre-trained perceptual features alone guarantee effective policy learning. ReWAM is built on pre-trained DINO features, organized into compact world states via Feature Calibration and a Temporal Representation Bottleneck suited to dynamics modeling. Action-Grounded Representation Shaping routes only action-loss gradients to the bottleneck, letting the policy shape what the representation encodes while the world model learns how it evolves. Notably, without generative video pre-training, ReWAM achieves a 93.6% success rate on RoboTwin 2.0 and, using roughly 600 hours of embodied pre-training data, reaches an average score of 12.29 with 8.28% success rate on RoboDojo. Paper: arXiv 2609.38163.

Paper Overview

Research Area: Computer Vision / Robotics Authors: Haoyi Jiang, Liu Liu, Xinjiang Wang, Zhihao Sun, Zequn Chen, Sen Wang, Xinjie Wang, Xia Chen, Jingfeng Yao, Weiheng Zhao, Shanglin Yuan, Zhizhong Su, Wei Sui, Wenyu Liu, Xinggang Wang Published: 2026-09-29 arXiv: 2609.38163

Abstract

World-action models jointly learn robot policies and predict future observations, making the representation space an interface between control and prediction. We study the design of this space through controlled comparisons, finding that neither reconstruction fidelity nor pre-trained perceptual features alone ensure effective policy learning.

These findings motivate ReWAM, a representation-centric world-action model built on pre-trained DINO features.

Key Components

Feature Calibration and a Temporal Representation Bottleneck organize pre-trained DINO features into compact world states suited to dynamics modeling.
Action-Grounded Representation Shaping routes only action-loss gradients to the bottleneck, thereby letting the policy shape *what* the representation encodes while the world model learns *how* it evolves.

Results

RoboTwin 2.0: 93.6% success rate, achieved without generative video pre-training.
RoboDojo: with approximately 600 hours of embodied pre-training data, an average score of 12.29 and an 8.28% success rate.

---

*Auto-collected on 2026-10-01*