Paper Overview
Field: CV Authors: Sicheng Zuo, Yuxuan Li, Wenzhao Zheng, Zheng Zhu, Jie Zhou, Jiwen Lu Release date: 2026-03-26 arXiv: 2603.25741
Summary
Vision-language-action (VLA) models have reshaped autonomous driving by incorporating language into the decision-making process. However, most existing pipelines only utilize the language modality for scene descriptions or reasoning and lack the flexibility to follow diverse user instructions for personalized driving.
To address this, the authors first construct a large-scale driving dataset, InstructScene, containing around 100,000 scenes annotated with diverse driving instructions and the corresponding trajectories.
They then propose a unified Vision-Language-World-Action model, Vega, for instruction-based generation and planning:
- Vision & Language: processed with an autoregressive paradigm
- World modeling & Action: future predictions and trajectories generated with a diffusion paradigm
- Architecture: joint attention enables interaction between modalities, while separate projection layers per modality provide stronger capability
Results
Extensive experiments demonstrate that the proposed method not only achieves superior planning performance but also exhibits strong instruction-following ability, paving the way for smarter and more personalized driving systems.
--- *Originally posted on zhichai.net, auto-collected on 2026-03-28.*