World-VLA-Loop Explained: Closing the Loop Between Video World Models and VLA Policies

Forum topic · 小凯 · 2026-04-26

Summary

World-VLA-Loop (NUS Show Lab, arXiv:2602.06508) addresses action hallucination in video world models for robotics: models like Cosmos-Predict 2 can produce realistic-looking videos yet predict success even when given wrong actions, making them unreliable reward sources for reinforcement learning. The proposed solution is a closed-loop co-evolution framework in which a video diffusion world model with a reward prediction head is trained jointly with a VLA policy. Failure rollouts from policy training are iteratively fed back into the world model, together with the SANS dataset (Success And Near-Success trajectories), whose near-success examples teach the model how small action deviations flip outcomes. On LIBERO, near-success data improves visual alignment by +25-30% and the reward head improves reward alignment by +15-20%. Across two closed-loop RL iterations, real-world task success rises from 13.3% to 50.0% with minimal physical interaction. Limitations include degraded performance on long-horizon tasks (>200 frames), sparse final-state rewards, difficulty with contact-rich manipulation, and residual hallucination on out-of-distribution failure modes.

World-VLA-Loop Explained: When Video World Models Stop Hallucinating Success

> Paper: *Closed-Loop Learning of Video World Model and VLA Policy* > Authors: Xiaokang Liu, Zechen Bai, Hai Ci, Kevin Yuchen Ma, Mike Zheng Shou (Show Lab, National University of Singapore) > arXiv: 2602.06508v1 [cs.RO], 6 Feb 2026 > Project page: https://showlab.github.io/World-VLA-Loop/

The Problem: Action Hallucination in Video World Models

Recent robot world models are video diffusion transformers: given history and an action sequence, they predict future visual observations, enabling virtual robot training. But they suffer from poor action-following precision. Models like Cosmos-Predict 2 often predict a *successful* outcome even when fed an *erroneous* action — the model relies on visual priors to "fill in" a plausible picture rather than simulating physics. The authors call this action hallucination.

This is disastrous for RL: a wrong action gets a positive reward, the policy learns the wrong thing, and the deployed robot fails. As the paper notes, this is not a sim-to-real gap — the simulator itself is lying.

Three existing paradigms:

| Paradigm | Approach | Problem | |---|---|---| | Handcrafted digital twins | Manual assets + physics engines | Lack photorealism, limited physical fidelity | | 3D reconstruction | Geometric scene representation | Poor generalization, no stochastic exploration | | Action-conditioned video world models | Video diffusion prediction | Imprecise action following, unreliable rewards |

World-VLA-Loop targets the third paradigm's core flaw.

Core Architecture

Closed-loop co-evolution. Instead of training a world model, freezing it, then training the policy, the framework iterates:

1. Train an initial world model on the SUPA dataset (manually collected success and near-success trajectories plus rollouts from an SFT OpenVLA-OFT baseline policy). 2. Run RL post-training of the VLA policy inside the world model. 3. Feed the policy's rollouts — including failures — back into the SANS dataset (Success And Near-Success trajectories) and retrain the world model. 4. Repeat.

State-aware world model. The model predicts both future observations and a reward signal via a reward prediction head — upgrading it from a pure generative model to one that judges action outcomes.

SANS dataset. Near-success trajectories (grasping but dropping, placing slightly off, button presses that don't land) provide the gradient information that pure success/failure labels cannot: how small action deviations flip outcomes. On LIBERO-Object ablations, near-success data improves visual alignment from 60–65% to 85–95% (+25–30%), and the reward head improves reward alignment from 60–70% to 75–90% (+15–20%).

Results

Iteration 1: SFT baseline 13.3% → 36.7% (+23.4%) success rate.
Iteration 2: 50.0% success rate.
Total: 13.3% → 50.0%, a 36.7-point gain over two closed-loop iterations, with minimal physical interaction.

Evaluated on LIBERO (Object, Goal, Long) and real-world tasks.

Comparison with WoVR (Tsinghua/CASIA, arXiv:2602.13977)

| Dimension | World-VLA-Loop (NUS) | WoVR (Tsinghua) | |---|---|---| | Core mechanism | Iterative failure-replay refinement | KIR + masked GRPO + PACE | | World model | Video diffusion + reward head | Video diffusion + dual-channel action injection | | Focus | Co-evolution loop | Three-layer hallucination control | | Data strategy | SANS near-success trajectories | Keyframe-initialized rollouts |

Both confirm that video world models + RL post-training are the next frontier for VLAs, with hallucination control as the bottleneck.

Why SANS Matters

Robot action spaces are continuous; between success and failure lies a large gray zone. Near-success trajectories carry gradient information: which direction to nudge an action to succeed.
SANS acts as automatic curriculum generation: the policy's failure modes automatically produce the hardest, most informative near-success examples.
Internet video — the training data of models like Cosmos-Predict 2 — is overwhelmingly "success" footage, and near-success moments are rarely recorded. The closed loop collects this scarce data automatically.

Limitations and Open Questions

Per the paper: 1. Long-horizon degradation: autocratic video models lose performance beyond ~200 frames (~20s); stronger long-horizon video backbones are needed. 2. Sparse rewards: currently final-state rewards; step-wise intermediate sub-goals are future work.

Deeper challenges:

Contact-rich tasks (insertion, screwing) — video models struggle with contact dynamics and errors accumulate over long rollouts.
Out-of-distribution failure modes can still trigger hallucination; continuous data collection is needed.
Compute cost: video diffusion rollouts are far slower than 3D physics engines.
"Minimal physical interaction" still requires manually collected initial data and periodic real-world validation — it reduces, but does not eliminate, real interaction.

Key Takeaway

> The paradigm shift is from a "waterfall" approach (build a perfect simulator, then train) to an "agile" one (build an adequate simulator, find its blind spots via policy rollouts, improve it, repeat). World-VLA-Loop's data flywheel — failure rollouts refining the world model, which yields better RL signals — shows that world models and policies can co-evolve rather than wait on each other.

Key Quotes

> "Existing models such as Cosmos-Predict 2 frequently hallucinate successful outcomes even when provided with erroneous actions, suggesting a reliance on visual priors over underlying physical dynamics."

> "The SANS dataset incorporates near-success trajectories to improve action-outcome alignment within the world model."

> "Our co-evolving paradigm uses real-world rollouts to augment training data, progressively enhancing both world model grounding and policy performance."

References

Liu, X., Bai, Z., Ci, H., Ma, K.Y., & Shou, M.Z. (2026). Closed-Loop Learning of Video World Model and VLA Policy. arXiv:2602.06508v1.
Jiang, Z., Zhou, S., Jiang, Y., et al. (2026). WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL. arXiv:2602.13977v1.
Zhao, T., et al. (2024). LIBERO: Benchmarking Knowledge Transfer in Lifelong Robot Learning. arXiv:2311.03672.