Agile-WAM: An Agile Tactile World Action Model for Contact-Rich Robot Control

Overview

Field: Machine Learning Authors: Hanchu Zhou, Brendan Lynch, Raman Goyal, Dechen Gao, Begum Kasap, Boqi Zhao, Junshan Zhang Published: 2026-09-17 arXiv: 2609.20761

Summary

World Action Models (WAMs) advance beyond conventional visuomotor policies by jointly predicting future world states and robot actions, enabling the policy to learn physical dynamics that support effective control. However, recent tactile WAMs often rely on large-scale pretrained generative backbones to capture contact-rich physical dynamics, which limits their inference efficiency and flexible deployment.

This paper presents Agile-WAM, an agile tactile World Action Model for contact-rich robot control. Agile-WAM encodes visual and tactile observations into a shared latent space that serves as the source of a direct vision-tactile-to-action flow-matching process, jointly generating latent representations of action chunks and future visual/tactile latents.

Key Insight: Multimodal Timescales

A key observation is that visual and tactile signals live at different timescales—adjacent visual frames are often highly similar, whereas tactile signals can change abruptly at the moment of contact. To address this, the authors introduce multi-horizon multimodal prediction: visual latent representations are supervised over longer time horizons, while tactile latents use next-frame prediction only, capturing fine-grained contact dynamics.

Results

Evaluated on 9 simulation tasks and 5 real-world contact manipulation tasks, Agile-WAM outperforms the strongest baselines while maintaining low inference latency:

• 29.4% relative improvement in overall success rate in real-world experiments
• 11.9 ms inference latency

These results demonstrate that multimodal WAMs can be built with lightweight architectures suitable for high-precision, high-frequency robot control.

Abstract (Original)

> World Action Models (WAMs) advance beyond conventional visuomotor policies by jointly predicting future world states and robot actions, enabling the policy to learn physical dynamics that support effective control. However, recent tactile WAMs often rely on large-scale pretrained generative backbones to capture contact-rich physical dynamics, which limit their inference efficiency and flexible deployment.

Full abstract: arXiv:2609.20761

---

*Auto-collected on 2026-09-20*