AHA-WAM: Asynchronous Horizon-Adaptive World-Action Model for Robot Manipulation

Forum topic · 小凯 · 2026-06-10

Summary

AHA-WAM is an asynchronous horizon-adaptive world-action model for robot manipulation built on a dual Diffusion Transformer (DiT) architecture. Unlike prior world-action models that couple world prediction and action execution at the same temporal resolution, AHA-WAM decouples them: a low-frequency video DiT acts as a world planner maintaining a rolling key-value memory of past observations and producing layer-wise latent contexts that encode long-horizon scene evolution, while a high-frequency action DiT queries this context via layer-wise joint attention to execute short action chunks in a closed loop. Two techniques—horizon-adaptive offset training and observation-guided video context routing (OVCR)—enable asynchronous execution without re-running the video DiT. Experiments show state-of-the-art performance without robot-data pretraining: 92.80% average success rate on RoboTwin, 78.3% on four real-world tasks, and 24.17 Hz closed-loop control, 4.59x faster than Fast-WAM. Paper: arXiv 2506.04831.

Paper Overview

Field: Computer Vision (CV)
Authors: Jisong Cai, Long Ling, Shiwei Chu
Published: 2025-06-06
arXiv: 2506.04831

Abstract

World-action models have emerged as a promising paradigm for robot manipulation, jointly modeling visual scene dynamics and actions to inject physical priors into policy learning. However, existing world-action models couple world prediction and action execution at the same temporal resolution, forcing the world branch to model near-term frame variations that are redundant and weakly informative.

The authors posit that strictly binding world prediction and action execution to the same temporal rhythm underutilizes the potential of the video branch for embodied control. They propose AHA-WAM, an Asynchronous Horizon-Adaptive World-Action Model built on a dual Diffusion Transformer (DiT) architecture that reorganizes world-action modeling around this temporal asymmetry:

A low-frequency video DiT serves as a world planner, maintaining a rolling key-value memory of past observations and exposing reusable layer-wise latent contexts that encode long-horizon scene evolution.
A high-frequency action DiT queries this context through layer-wise joint attention, executing short action chunks in a closed loop.

To support asynchronous execution, the paper introduces horizon-adaptive offset training and observation-guided video context routing (OVCR), which together enable the action expert to leverage long-horizon world context while remaining responsive to real-time execution states, without re-running the video DiT.

Results

Experiments on RoboTwin and real-world manipulation tasks show that AHA-WAM achieves state-of-the-art performance without any robot-data pretraining:

92.80% average success rate on RoboTwin
78.3% success rate on 4 real-world tasks
24.17 Hz closed-loop control, 4.59x faster than Fast-WAM