LaST-R1 Explained: Teaching Robots to Imagine Physical Futures Before Acting

Forum topic · 小凯 · 2026-05-03

Summary

LaST-R1 is a robotics research paper from The Chinese University of Hong Kong, Peking University, and Simplexity Robotics (arXiv:2604.28192) that addresses the tension between reasoning and control in Vision-Language-Action (VLA) models. Instead of reasoning in slow, discrete language tokens, the model produces a latent chain-of-thought grounded in predicted future visual states from a vision foundation model, acting as a physical 'mental rehearsal' before action generation. Its core algorithm, Latent-to-Action Policy Optimization (LAPO), applies reinforcement learning jointly over latent reasoning tokens and action tokens, treating reasoning as implicit decision variables. An adaptive mechanism lets the model emit a <latent_end> token to adjust reasoning length to task complexity. Results are striking: 99.8% average success on the LIBERO benchmark, 99.4% on the hardest LIBERO-Long suite with only one demonstration per task, and a 90% average success rate in real-world single-arm and dual-arm manipulation with up to 44% improvement from RL post-training and zero-shot generalization to unseen objects and conditions. Limitations include a small 4B backbone, limited real-world task coverage, poor interpretability of latent thoughts, and reliance on clear reward signals.

LaST-R1: Reinforcing Action via Adaptive Physical Latent Reasoning for VLA Models

Authors: Chenyang Gu, Jialin Gao, Ziyu Guo, Siyuan Qian, Yinxi Wang, Peng Jia, Chi-Wing Fu, Shanghang Zhang, Pheng-Ann Heng Institutions: The Chinese University of Hong Kong, Peking University, Simplexity Robotics arXiv: 2604.28192 | Project page: https://siriyep.github.io/last-r1/

Key points

The problem: reasoning vs. control in VLA models

Vision-Language-Action models (RT-2, OpenVLA, π₀) map vision + language instructions directly to robot actions, but face a tension: reasoning needs time, control needs speed.
Explicit linguistic reasoning (generating text like "move left, then grip") is interpretable but slow and crudely quantizes continuous physical trajectories.
Existing latent reasoning is more expressive but trapped in static imitation learning — it copies demonstrations without exploration or adaptation.
Existing RL for VLAs optimizes only the action space, never the reasoning process itself.

Innovation 1: Physically grounded latent chain-of-thought

LaST-R1 autoregressively generates latent reasoning tokens explicitly anchored on global future representations from a vision foundation model (VPT — Visual Proprioceptive Transformer).
The latent tokens encode an imagined future — object trajectories, contact points, forces — serving as a conditioning signal for action generation. It is a non-verbal physical "mental rehearsal."

Innovation 2: LAPO — Latent-to-Action Policy Optimization

LAPO treats latent reasoning tokens as implicit decision variables in RL. Reward flows back to update both the reasoning process and action generation.
When a grasp fails, LAPO can attribute the failure to faulty physical reasoning (e.g., misjudged mass) and correct the *imagination*, not just the trajectory.
It computes a joint step-level likelihood ratio over latent and action tokens:

∇J = E[ Σ (advantage_t × ∇log π(latent_t, action_t | state_t)) ]

Joint optimization creates a feedback loop: better reasoning → better actions → reinforced reasoning.

Innovation 3: Adaptive reasoning horizon

An adaptive latent chain-of-thought mechanism emits a special <latent_end> token when "thought is sufficient." Simple reactive tasks get short reasoning; complex multi-step manipulation gets longer deliberation.
The optimal reasoning length is learned through RL, not manually programmed.

Results

LIBERO benchmark: 99.8% average success rate across all four suites (Spatial, Object, Goal, Long).
LIBERO-Long: 99.4% with only one demonstration per task, vs. ~97% for previous SOTA with full expert datasets.
Convergence: LAPO reaches near-optimal performance in far fewer steps than PPO baselines; latent reasoning acts as a "cognitive buffer" easing credit assignment.
Real-world: up to 44% improvement over the warm-up policy from LAPO post-training; 90% average success on complex single-arm and dual-arm tasks (precise insertion, tool use, articulated objects), with zero-shot generalization to unseen objects, backgrounds, and lighting.
Architecture: SigLIP2-Large visual encoder with 2D-RoPE; Qwen3-VL-4B backbone; action tokenizer preserving physical meaning.

Honest critique

Scale: Qwen3-VL-4B backbone is small; scaling to 70B+ is undemonstrated, and autoregressive latent generation raises real-time deployment cost concerns.
Sim-to-real gap: real-world validation covers only four controlled tasks.
Interpretability: latent tokens are expressive but not human-readable — we can't ask the robot what it was thinking.
Reward problem: LAPO needs clear task success/failure signals; ambiguous or delayed rewards remain open.

Why it matters

Industrial robotics: one-shot warm-up plus RL refinement could cut task programming from months to hours.
Home robotics: physical reasoning plus trial-and-error adaptation could handle unstructured household environments.
AI research: success of latent reasoning suggests non-linguistic, continuous-space thinking may transfer to other domains — challenging the assumption that language is the only medium of thought.

References

Gu, C., et al. (2026). LaST-R1: Reinforcing Action via Adaptive Physical Latent Reasoning for VLA Models. arXiv:2604.28192.
Project page: https://siriyep.github.io/last-r1/
Liu, Z., et al. (2026). LaST₀: Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Model. arXiv:2601.05248.
Liu, J., et al. (2025). What Can RL Bring to VLA Generalization? An Empirical Study. arXiv:2505.19789.
Schulman, J., et al. (2017). Proximal Policy Optimization Algorithms. arXiv:1707.06347.