Learning Action Priors for Cross-embodiment Robot Manipulation

Forum topic · 小凯 · 2026-06-26

Summary

This post introduces an arXiv paper (2606.19233) by Dong Jing, Tianqi Zhang, and Jiaqi Liu on improving Vision-Language-Action (VLA) models for robot manipulation. Most VLA models attach an action module to a Vision-Language Model (VLM) backbone and optimize the full policy jointly, which means the action module must learn physical motion almost from scratch. The authors propose a two-stage training framework that first pretrains the action module with motion priors before cross-modal VLA alignment. In stage one, a lightweight flow-matching encoder-decoder learns temporal motion structure purely from unconditional action trajectories, cross-embodiment and without visual or language tokens. In stage two, this prior is transferred to VLA training via decoder reuse and early latent distillation, aligning vision-language features with the action embedding space while still allowing end-to-end optimization. The trained encoder also compresses state-action history into a single temporal context token, enabling history-aware modeling at low cost. Experiments on 13 cross-embodiment tasks in simulation and on real robots show faster convergence and higher success rates than VLA training without action priors, with notably stronger performance on data-scarce real-world tasks. Scaling stage-one action data yields more generalizable priors that directly improve downstream VLA performance.

Paper Overview

Field: Robotics Authors: Dong Jing, Tianqi Zhang, Jiaqi Liu Published: 2026-06-25 arXiv: 2606.19233

Abstract

Most Vision-Language-Action (VLA) models build on a Vision-Language Model (VLM) backbone by attaching an action module and optimizing the full policy jointly. This design inherits strong visual and linguistic priors from the VLM, but leaves the action module to learn physical motion almost from scratch. As a result, the policy lacks an explicit motion prior, forcing early optimization to simultaneously discover temporal action dynamics and cross-modal alignment — a challenge further amplified in cross-embodiment settings.

Key Idea

The authors propose pretraining the action module with motion priors *before* cross-modal VLA alignment, using a two-stage training framework that equips the action module with cross-embodiment temporal motion structure before VLA training begins.

Method

Stage 1 — Motion prior learning: A lightweight flow-matching-based encoder-decoder action module efficiently learns temporal motion structure solely from unconditional action trajectories, without processing visual or language tokens.
Stage 2 — Transfer to VLA training: The learned prior is transferred via decoder reuse and early latent distillation, aligning vision-language features with the action embedding space while still permitting end-to-end policy optimization.
History compression: The trained encoder serves as a compact history compressor, summarizing state-action history into a single temporal context token, enabling history-aware modeling at minimal cost.

Results

Evaluated on 13 diverse cross-embodiment tasks in both simulation and on real platforms.
Compared with VLA training without action priors, the model converges faster and achieves higher success rates.
Performance is significantly stronger on data-scarce real-world tasks.
Scaling stage-one action data yields more generalizable action priors, directly improving downstream VLA performance.

---

*Auto-collected on 2026-06-26.*