[论文] VITAL: From Visual Motion to Precise Action

论文概要

研究领域: CV 作者: Ao Liang, Youquan Liu, Mingxiao Li, Wei Chow, Lingdong Kong 发布时间: 2026-10-14 arXiv: 2610.20604

中文摘要

世界-动作模型旨在将视觉预判转化为驾驶决策，但通常在与视觉动力学无关定义的轨迹表示上训练动作专家。这促使我们直接从视觉运动中导出动作目标。我们首先考察一个更简单的场景：从记录的未来图像中重建轨迹。即使不需要未来预测，重建也能捕获主要机动动作，但会留下厘米级误差。因此我们提出VITAL（视觉引导的动作学习token），一种动作tokenizer，将从视觉导出的运动支架与该支架留下的轨迹误差的紧凑残差编码相结合。这种表示使视觉运动成为主要预测目标，并为小的度量修正提供了独立的归一化学习尺度。使用流匹配，动作专家从当前图像、自车状态和导航指令联合预测两个分量，推理时无需生成未来图像。同一残差坐标支持有界强化后训练，在保持预测视觉支架固定的同时精炼执行。仅用一个动作去噪步，VITAL在NAVSIM v1 navtest上达到90.8 PDMS，在3.29倍端到端推理速度下与十步得分仅差0.13分。无需重新训练，它在NAVSIM v2 navtest上达到91.1 EPDMS，在navhard上达到38.6。

原文摘要

World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined independently of visual dynamics. This motivates deriving action targets directly from visual motion. We first examine a simpler setting in which trajectories are reconstructed from recorded future images. Even without the need for future prediction, reconstruction captures the dominant maneuver but leaves centimeter-scale errors. We therefore introduce VITAL (Vision-Induced Tokens for Action Learning), an action tokenizer that couples a visually derived motion scaffold with a compact residual encoding the trajectory error left by that scaffold. This representation makes visual motion the principal prediction target and gives small metric corr...

#论文 #arXiv #CV #小凯