[论文] VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding

论文概要

研究领域: CV
作者: Yu-Cheng Chou, Shihao Wang, Jin Huang, Chi-Pin Huang, Chiao-An Yang, Xianzhe Fan, Jing Wang, Zhiqi Li, Yuanguo Kuang, Ruijie Zheng, Johan Bjorck, Sifei Liu, Hang Su, Abhishek Badki, Lei Zhang, Alan Yuille, Linxi "Jim" Fan, Jan Kautz, Zhiding Yu
发布时间: 2026-11-05
arXiv: 2611.08449

中文摘要

面向机器人和自动驾驶等物理 AI 任务的模型，不必仅靠语言推理。它们可以使用视觉计划——点、边界框和轨迹——来表示空间目标和预测运动。然而，自回归视觉-语言模型逐坐标 token 生成这些计划，使得更丰富的预测成本越来越高。我们提出 VFlash，一种通过 Flash Primitive Decoding 实现快速、精确视觉规划的方法。VFlash 利用视觉基元的已知结构来联合预测其坐标，使用轻量级 drafter 在一次前向传播中生成整个基元。布局自适应适配基元的类型、维度和长度，支持紧凑的边界框和长的可变长度轨迹。关键在于，空间精度并不需要复现自回归模型的精确 token 序列。因此 VFlash 直接在标注基元上训练，无需目标 token 一致即可接受结构有效的预测。在评估的 2D 和 3D 机器人轨迹任务上，VFlash 相比自回归解码持续降低了预测误差。初步单 rollout 测量显示，3D 点运动预测生成速度约快 25 倍。以 VFlash 计划为条件的世界动作模型在四个 MolmoSpaces 任务套件上平均成功率为 70.2%，而基线为 62.6%。

原文摘要

Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.

自动采集于 2026-11-08

#论文 #arXiv #CV #小凯