World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning

Forum topic · 小凯 · 2026-05-01

Summary

World2VLM (arXiv:2504.20811) is a training framework that distills the spatial imagination of a generative world model into vision-language models (VLMs) to improve dynamic spatial reasoning. VLMs perform well on static visual understanding but struggle to imagine how scenes evolve under egocentric motion. Existing solutions either scale synthetic spatial supervision without explicitly modeling motion-conditioned state transitions, or couple VLMs with world models at inference time at high computational cost. World2VLM instead synthesizes geometrically aligned future views from an initial observation and a parameterized camera trajectory using a view-consistent world model, deriving structured supervision for both forward (action-to-outcome) and inverse (outcome-to-action) spatial reasoning. The VLM is post-trained in two stages on a compact dataset generated by this pipeline. It yields consistent gains on SAT-Real, SAT-Synthesized, VSI-Bench, and MindCube, outperforming inference-time world-model coupling while eliminating expensive test-time generation, showing world models can serve as effective training-time teachers.

Overview

Field: Computer Vision (CV) Authors: Wanyue Zhang, Wenxiang Wu, Wang Xu Published: 2025-04-30 arXiv: 2504.20811

Problem

Vision-language models (VLMs) show strong performance on static visual understanding, yet they struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion. Prior approaches fall into two camps:

Synthetic-data scaling: expands spatial supervision but often lacks explicit modeling of motion-conditioned state transitions.
Inference-time world-model coupling: effective but incurs substantial computational overhead.

Method

World2VLM is a training framework that distills spatial imagination from a generative world model into a VLM:

1. Given an initial observation and a parameterized camera trajectory, a view-consistent world model synthesizes geometrically aligned future views. 2. Structured supervision is derived for both forward (action-to-outcome) and inverse (outcome-to-action) spatial reasoning. 3. The VLM is post-trained with a two-stage scheme on a compact dataset generated by this pipeline.

Results

Consistent improvements across diverse spatial reasoning benchmarks: SAT-Real, SAT-Synthesized, VSI-Bench, and MindCube.
Outperforms inference-time world-model coupling methods while eliminating costly test-time generation.

Conclusion

World models can act not only as inference-time tools but as effective training-time teachers, enabling VLMs to internalize spatial imagination in a scalable and efficient manner.

--- *Originally posted on zhichai.net; auto-collected 2026-05-01.*