Long-WAM: Scaling the Context of World-Action Models for Real-Time Robot Control

Forum topic · 小凯 · 2026-10-09

Summary

Long-WAM (arXiv:2610.10528) is a model-system framework for scaling the context of causal world-action models under real-time robot control constraints. Its central finding is that access to visual history is not the same as using it: longer histories yield far greater benefits when the video foundation model is pretrained autoregressively (AR). The framework first learns causal prediction from robot and egocentric videos without action labels, then preserves this history-to-future structure during world-action adaptation. On RoboCasa GR-1, extending context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, while bidirectionally pretrained initialization shows no net gain. Long-WAM achieves state-of-the-art results on LIBERO-Long, RoboTwin 2.0, and DOMINO. Streaming observation encoding, asynchronous execution, and hardware-specific acceleration enable deployment on RTX 5090, DGX Spark, and Jetson AGX Thor, with each action chunk taking 107.4 ms on RTX 5090. Real-time deployment on Unitree G1 and YAM supports dynamic long-horizon manipulation, including a dynamic cup-stacking task at 95% success where Pi0.5 and Fast-WAM failed in all 20 trials.

Paper Overview

Research Area: Computer Vision Authors: Wei Huang, Bohan Zhang, Chenzhi Liu, Isabella Liu, Shuai Yang, Weian Mao, Luozhou Wang, Yicheng Xiao, Weifeng Lin, Qixin Hu, Bryan Chu, Sifei Liu, Linxi Fan, Xiaojuan Qi, Song Han, Yukang Chen Released: 2026-10-07 arXiv: 2610.10528

Abstract (Translated)

Real-time robot control demands enough visual history to infer motion states and task progress, but processing that history can delay action execution. The authors present Long-WAM, a model-system framework for scaling the context of causal world-action models under real-time control constraints.

The central finding: access to history is not the same as using it — longer histories pay off far more when the video foundation model is pretrained autoregressively (AR). The framework first learns causal prediction from robot and egocentric videos without action labels, then preserves this history-to-future structure during world-action adaptation.

Key Results

On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%; a bidirectionally pretrained initialization shows no net gain.
AR pretraining in the robot domain further improves peak success on GR-1 and LIBERO-Long.
Long-WAM achieves best results on LIBERO-Long, RoboTwin 2.0, and DOMINO.
Streaming observation encoding, asynchronous execution, and hardware-specific acceleration enable deployment on RTX 5090, DGX Spark, and Jetson AGX Thor without dropping future prediction; each action chunk (including future video latent prediction) takes 107.4 ms on RTX 5090.
Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including a dynamic cup-stacking task at 95% success — Pi0.5 and Fast-WAM failed in all 20 trials.
As a memory-driven actuator, Long-WAM also complements high-level planners on composite tasks.

Original Abstract (Excerpt)

> Real-time robot control demands enough visual history to infer motion and task progress, but processing that history can delay action. We present Long-WAM, a model-system framework for scaling the context of causal world-action models under real-time control constraints. Our central finding is that access to history is not the same as using it: longer histories pay off far more when the video foundation is pretrained autoregressively (AR). We first learn causal prediction from robot and egocentric videos without action labels, then preserve this history-to-future structure during world-action adaptation. On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, whereas a bidirectionally pretrained initialization shows no net gain; robot-domain AR pre...

--- *Auto-collected on 2026-10-09*