iMaC: Translating Actions into Motion and Contact Images for Embodied World Models

Forum topic · 小凯 · 2026-06-10

Summary

iMaC (Image as Action Control) is a unified control paradigm for embodied world models that replaces low-dimensional structured action vectors (such as joint angles and end-effector poses) with raw visual images as native action representations. Traditional embodied frameworks suffer from limited expressive capacity, poor generalization across diverse embodiments, and unnatural dynamic modeling of complex physical interactions. iMaC formulates continuous visual manipulation as image-based action tokens that inherently encapsulate spatial motion intent, interaction geometry constraints, and subtle physical dynamics. The authors build a dual-branch embodied architecture consisting of an image action encoder, which compresses goal-driven visual images into compact action embeddings, and a dynamics world predictor, which learns environment transition rules conditioned on image actions for high-fidelity future state prediction and closed-loop embodied control. Experiments on public embodied manipulation benchmarks and real-world robot scenarios show that iMaC outperforms vector-based action control baselines in prediction accuracy, task success rate, and cross-scenario generalization, while eliminating the need for manually defined action spaces and enabling flexible universal control of heterogeneous embodied agents. Paper: arXiv 2506.04834, published June 6, 2025.

Paper Overview

Research Area: Computer Vision (CV) Authors: Zhenyu Wu, Xiuwei Xu, Yukun Zhou Published: 2025-06-06 arXiv: 2506.04834

Abstract

Embodied world models have emerged as a pivotal paradigm for visual robotic decision-making and interactive environment simulation. However, conventional embodied frameworks rely on low-dimensional structured action vectors (e.g., joint angles and end-effector poses), which suffer from limited expressive capacity, poor generalization across diverse embodiments, and unnatural dynamic modeling for complex physical interactions.

To address these limitations, this paper proposes iMaC (Image as Action Control), a novel unified control paradigm that treats raw visual images as native action representations for embodied world models. Departing from traditional explicit kinematic action encoding, iMaC formulates continuous visual manipulation as image-based action tokens, which inherently encapsulate spatial motion intent, interaction geometry constraints, and subtle physical dynamics.

Architecture

The authors construct a dual-branch embodied architecture:

Image action encoder: compresses goal-driven visual images into compact action embeddings
Dynamics world predictor: learns environment transition rules conditioned on image actions, enabling high-fidelity future state prediction and closed-loop embodied control

Key Findings

Extensive experiments on public embodied manipulation benchmarks and real-world robot scenarios show that iMaC outperforms vector-based action control baselines in prediction accuracy, task success rate, and cross-scenario generalization.
The image-based action design removes the dependence on manually defined action spaces, enabling flexible and universal control of heterogeneous embodied agents.