Paper Overview
Field: Machine Learning Authors: Hanchu Zhou, Brendan Lynch, Raman Goyal, Dechen Gao, Begum Kasap, Boqi Zhao, Junshan Zhang Published: 2026-09-17 arXiv: 2609.20761
Summary
World Action Models (WAMs) go beyond conventional visuomotor policies by jointly predicting future world states and robot actions, allowing the policy to learn the physical dynamics that support effective control. However, recent tactile WAMs often rely on large-scale pretrained generative backbones to capture contact-rich physical dynamics, which limits inference efficiency and flexible deployment.
This paper presents Agile-WAM, an agile tactile World Action Model for contact-rich robot control. Agile-WAM encodes visual and tactile observations into a shared latent space, which serves as the source of a direct vision-tactile-to-action flow-matching process that jointly generates latent representations of action chunks and future visual/tactile latents.
Key Insight
Vision and tactile signals operate on fundamentally different timescales: adjacent visual frames tend to be highly similar, whereas tactile signals can change abruptly at the moment of contact.
Multi-Horizon Multi-Modal Prediction
To exploit this observation, the authors introduce multi-horizon multi-modal prediction:
- Visual latents receive supervision over a longer time horizon.
- Tactile latents are supervised only with next-frame prediction, capturing fine-grained contact dynamics.
- 29.4% relative improvement in overall real-world success rate
- 11.9 ms inference latency
Results
Evaluated on 9 simulated tasks and 5 real-world contact-rich manipulation tasks, Agile-WAM achieves strong performance, surpassing the strongest baselines while maintaining low inference latency:
---
*Auto-collected on 2026-09-20.*