LaST-VLA: Physics-Grounded Latent Spatio-Temporal Reasoning for Autonomous Driving VLA Models

Forum topic · 小凯 · 2026-04-27

Summary

LaST-VLA, developed by researchers from Tsinghua University, Xiaomi, and the University of Macau, replaces text-based chain-of-thought reasoning in Vision-Language-Action (VLA) models with physics-grounded latent spatio-temporal reasoning for autonomous driving. The method distills geometric knowledge from a 3D foundation model (Cosmos/VGGT) and dynamic foresight from a video world model into two decoupled latent channels. A structured causal mask with a visual bottleneck forces all visual information to pass through the latent thinking stage. Training proceeds in three phases: physics-aware alignment, latent-grounded planning, and GRPO-based reinforcement refinement. LaST-VLA achieves a record 91.3 PDMS on NAVSIM v1 and 87.1 EPDMS on NAVSIM v2, outperforming DriveVLA-W0 and RecogDrive. Notably, the 2B variant reaches 91.1 PDMS, nearly matching the 8B model, showing that physics alignment matters more than scale. The model also leads on spatial reasoning (SURDS) and dynamic scene understanding (NuDynamics) benchmarks. Ablations confirm that supervised latent CoT (91.3) outperforms unsupervised latent CoT (89.8) and text CoT (87.2), demonstrating that explicit physical grounding is the decisive factor.

LaST-VLA Deep Dive: Giving Autonomous Driving AI Physical 'Spatial Intuition'

TL;DR: LaST-VLA (Tsinghua + Xiaomi + University of Macau) replaces text chain-of-thought with a continuous latent spatio-temporal reasoning space grounded in physics, giving driving AI spatial intuition and foresight — scoring a record 91.3 on NAVSIM v1.

Key points

Text CoT in VLA models suffers from semantic-perception decoupling, hallucination, and high latency; naive latent CoT lacks physical grounding.
LaST-VLA aligns latent reasoning with physics via dual feature alignment: a geometry adapter distills from a 3D foundation model (Cosmos/VGGT), a dynamics adapter distills from a video world model. Latent states are decoupled into dynamic features (H_d) and geometric features (H_g).
A structured causal mask includes mutual masking between 3D/world-model tokens and a visual bottleneck that prevents action tokens from attending directly to raw images — forcing decisions through latent thinking.

Training: learn physics first, then drive

1. Phase I — Physics-aware alignment: loss weights λ_WM = λ_3D = 1.0 ≫ λ_action = 0.01; the latent CoT strictly aligns with teacher geometry/dynamics. 2. Phase II — Latent-grounded planning: weights flip (λ_action = 1.0); action tokens may also view raw images to refine driving policy. 3. Phase III — GRPO refinement: geometry/dynamics adapters frozen; reward combines PDMS trajectory quality, format compliance, and goal precision.

Results

NAVSIM v1 (PDMS)

| Model | PDMS | NC | DAC | TTC | EP | |---|---|---|---|---|---| | LaST-VLA-8B | 91.3 | 98.7 | 97.9 | 95.6 | 86.7 | | LaST-VLA-2B | 91.1 | 98.6 | 97.7 | 95.8 | 86.4 | | DriveVLA-W0-7B | 86.1 | 96.5 | 94.0 | 94.1 | 80.0 | | Recogdrive-2B | 90.8 | 98.3 | 94.7 | 95.5 | 83.0 |

NAVSIM v2 (EPDMS): LaST-VLA-8B scores 87.1 (SOTA), 2B scores 86.8.

Spatial reasoning (SURDS, 8B vs InternVL3-8B)

Yaw: 70.16% vs 54.50% (+28.7%)
Pixel localization: 71.28% vs 56.67% (+25.8%)
Depth range: 62.14% vs 53.64%
Left/right: 90.27% vs 82.71%

Dynamic scenes (NuDynamics): 81.19% for 8B — beating even the 72B Qwen2.5-VL (74.50%).

Ablations

Geometry only: 90.1 PDMS; dynamics only: 90.0; both: 91.3 vs baseline 87.9.
Latent CoT unsupervised (89.8) already beats text CoT (87.2); supervised physics alignment pushes it to 91.3 — grounding is the decisive factor.

Significance

Latent reasoning's potential lies not in dropping text, but in injecting physical grounding into latent space.
Scale is not the deciding factor: the 2B model nearly matches 8B.
Gains in NC (no at-fault collision) and DAC (drivable area compliance) directly translate to real-world safety.

Limitations: validation mainly on NAVSIM (OpenScene-derived); latent reasoning is less interpretable than text; teacher model quality affects distillation.

References

Paper: *LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving* (arXiv: 2603.01928v2), by Yuechen Luo, Fang Li, Shaoqing Xu et al.
Code: https://github.com/luo-yc17/LaST-VLA
Chinese analysis: https://m.sohu.com/a/997270395_115479
Related: DriveWorld-VLA (arXiv: 2602.06521v1)