DeCAL: Contact-Aware Latent Co-Imagination for Physically-Grounded Dexterous Vision-Language-Action Models

Forum topic · 小凯 · 2026-09-10

Summary

DeCAL is a physically-grounded dexterous vision-language-action (VLA) model designed for contact-rich robotic manipulation, where severe visual occlusions and complex contact dynamics challenge existing approaches. Built on a Mixture-of-Transformers (MoT) architecture, DeCAL assigns specialized experts to understanding, imagination, and action generation while maintaining efficient information flow between them. The model introduces two key innovations: Adaptive Visuo-Tactile Fusion, which dynamically regulates tactile integration through a contact-aware gating strategy, and Visuo-Tactile Latent Co-Imagination, which jointly models visual and tactile dynamics to equip the policy with implicit physical world knowledge. Experiments show DeCAL achieves state-of-the-art performance across all tasks with a 71% average success rate and an 83.4% progress success rate, along with strong generalization to unseen scenarios. The paper (arXiv:2609.09119) is authored by Yankai Fu, Ning Chen, Junkai Zhao, and colleagues from cs.RO/cs.AI.

Paper Overview

Research areas: cs.RO, cs.AI
Authors: Yankai Fu, Ning Chen, Junkai Zhao, Heng Zhang, Guocai Yao, Pengwei Wang, Zhongyuan Wang, Shanghang Zhang
Published: 2026-09-08
arXiv: 2609.09119

Abstract

Dexterous manipulation involves contact-rich and fine-grained interactions with the physical world, posing significant challenges for existing vision-language-action (VLA) models due to severe visual occlusions and complex contact dynamics. While recent works have incorporated tactile sensing into robotic manipulation, most approaches still rely on homogeneous multimodal fusion, lacking adaptive tactile integration and explicit modeling of physical dynamics.

In this work, the authors present DeCAL, a physically-grounded dexterous vision-language-action model that unifies understanding, imagination and action generation for contact-rich dexterous manipulation.

Key Contributions

Mixture-of-Transformers (MoT) architecture: DeCAL leverages specialized experts for each capability (understanding, imagination, action generation) while enabling efficient information flow among them.
Adaptive Visuo-Tactile Fusion: dynamically regulates tactile interactions via a contact-aware gating strategy to make effective use of tactile information.
Visuo-Tactile Latent Co-Imagination: jointly models visual and tactile dynamics, equipping the policy with implicit physical world knowledge.

Results

Experimental results show that DeCAL consistently achieves state-of-the-art performance across all tasks:

71% average success rate
83.4% progress success rate
Strong generalization to unseen scenarios