Action Images: End-to-End Policy Learning via Multiview Video Generation (arXiv 2504.06262)

Paper Overview

Field: Computer Vision (CV) Authors: Haoyu Zhen, Zixian Gao, Qiao Sun Published: 2025-04-08 arXiv: 2504.06262

Summary

World action models (WAMs) have emerged as a promising direction for robot policy learning, as they can leverage powerful video backbones to model future states. However, existing approaches often rely on separate action modules, or use action representations that are not pixel-grounded, making it difficult to fully exploit the pretrained knowledge of video models and limiting transfer across viewpoints and environments.

This work presents Action Images, a unified world action model that formulates policy learning as multiview video generation. Instead of encoding control as low-dimensional tokens, the authors translate 7-DoF robot actions into interpretable action images: multi-view action videos that are grounded in 2D pixels and explicitly track robot-arm motion.

Key Contributions

• A pixel-grounded action representation that converts 7-DoF robot control into multi-view action videos
• The video backbone itself acts as a zero-shot policy, eliminating the need for a separate policy head or action module
• Better utilization of pretrained video-model knowledge, with improved transfer across viewpoints and environments

Results

• Strongest zero-shot success rates on RLBench and real-world evaluations
• Surpasses prior video-space world models in joint video-action generation quality

The results indicate that interpretable action images are a promising path for policy learning.

--- *Auto-collected on 2026-04-09.*