Paper Overview
- Field: Computer Vision (CV)
- Authors: Junke Wang, Qihang Zhang, Shuai Yang
- Published: 2025-06-13
- arXiv: 2506.10666
- Paper: https://arxiv.org/abs/2506.10666
- Code and weights: https://github.com/wdrink/RepWAM
Abstract
This work presents RepWAM, a representation-centric world action model (WAM) built on representation visual-action tokenizers. Existing WAMs typically inherit reconstruction-oriented video tokenizers from pretrained video generation models. Although these tokenizers preserve visual fidelity, pixel reconstruction alone provides limited guidance for learning instruction-following dynamics that connect future prediction with robot control.
To address this, the authors explore a semantic visual-action latent space for representation-centric world action modeling. Specifically:
1. A representation visual-action tokenizer is trained to map visual inputs into aligned visual and latent action tokens. 2. The WAM is pretrained to jointly model future visual states and the latent actions that connect them, conditioned on language instructions. 3. The model is then adapted to real robot trajectories for closed-loop manipulation.
Experiments on real-world manipulation tasks and simulation benchmarks show that RepWAM performs strongly across various manipulation settings. Ablation studies highlight the value of semantic visual-action tokenization compared with reconstruction-oriented alternatives. These results establish representation visual-action tokenization as a promising foundation for world action models and a step toward generalist robot policies.