GeoThinker-F: Predictive Latent World Representations for Fast Spatial Reasoning in MLLMs

Forum topic · 小凯 · 2026-10-16

Summary

GeoThinker-F is an efficient framework for spatial reasoning in multimodal large language models (MLLMs) that removes the need for external 3D geometry encoders at inference time. Instead of running costly encoder paths, it converts causal MLLM hidden states into predictive latent world representations, based on the insight that intermediate semantic features contain enough information to predict useful representations of current geometry and future semantics without exactly replicating a dedicated 3D encoder. The framework uses a decoupled two-stage training strategy: first, with the MLLM backbone frozen, a lightweight Latent Frame Prediction (LFP) head learns from offline 3D geometry experts and future-frame semantic targets; second, the external expert is removed and the model jointly optimizes via a Spatial-Grounded Temporal (SGT) fusion module that generates and selectively integrates self-predicted priors. Evaluations show GeoThinker-F achieves up to 3.6x inference speedup, reducing latency from 1.45s to 0.40s while maintaining competitive performance across spatial intelligence benchmarks. Paper: arXiv 2610.20592.

GeoThinker-F: Endogenous Predictive World Representations for Fast Spatial Reasoning

Research area: Computer Vision (CV)
Authors: Haoyuan Li, Qihang Cao, Huanyuan Zhou, Xin Hu, Sijia Chen
Published: 2026-10-14
arXiv: 2610.20592

Abstract

Existing geometry-aware Multimodal Large Language Models (MLLMs) increasingly rely on external 3D encoders to provide structural priors, but executing these additional encoder paths at inference introduces substantial latency and computational overhead. GeoThinker-F is an efficient framework that internalizes spatiotemporal perception by transforming causal MLLM hidden states into predictive latent world representations.

The key insight: intermediate semantic representations contain sufficient information to predict useful representations of current geometry and future semantics, without requiring an exact reproduction of features from a dedicated 3D encoder.

Training Strategy

GeoThinker-F adopts a decoupled two-stage training strategy:

1. Stage 1: With the MLLM backbone frozen, a lightweight Latent Frame Prediction (LFP) head learns from offline 3D geometry experts and future-frame semantic targets. 2. Stage 2: The external expert is removed, and the model jointly optimizes via a dedicated Spatial-Grounded Temporal (SGT) fusion module that generates and selectively integrates its own predicted priors.

Results

Extensive evaluations show that GeoThinker-F eliminates the runtime dependency on external 3D geometry encoders while maintaining competitive performance on diverse spatial intelligence benchmarks. It achieves:

Up to 3.6x inference speedup
Latency reduced from 1.45s to 0.40s
Strong spatial reasoning performance retained