Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction

Forum topic · 小凯 · 2026-04-12

Summary

Scal3R is a computer vision paper (arXiv:2504.07865, published April 10, 2025) by Tao Xie, Peishan Yang, and Yudong Jin that tackles large-scale 3D scene reconstruction from long video sequences. Feed-forward reconstruction models regress 3D geometry directly from RGB images without explicit 3D priors or geometric constraints, but they struggle to maintain accuracy and consistency over long sequences due to limited memory capacity and weak global contextual cues. Inspired by how humans use global scene understanding to guide local perception, the authors introduce a novel neural global context representation that compresses and retains long-range scene information. This representation is implemented with lightweight neural sub-networks adapted at test time via self-supervised objectives, greatly expanding memory capacity with minimal computational overhead. Experiments on large-scale benchmarks including KITTI Odometry and Oxford Spires demonstrate leading pose accuracy and state-of-the-art 3D reconstruction accuracy while remaining efficient, validating the approach for very large scenes.

Paper Overview

Field: Computer Vision (CV)
Authors: Tao Xie, Peishan Yang, Yudong Jin
Published: 2025-04-10
arXiv: 2504.07865

Problem

The paper addresses large-scale 3D scene reconstruction from long video sequences. Recent feed-forward reconstruction models regress 3D geometry directly from RGB images without explicit 3D priors or geometric constraints, showing promising results. However, these methods struggle to maintain reconstruction accuracy and consistency over long sequences due to:

Limited memory capacity
Inability to effectively capture global contextual cues

Approach

Motivated by the observation that humans naturally exploit global scene understanding to inform local perception, the authors propose a novel neural global context representation that efficiently compresses and retains long-range scene information. This enables the model to leverage extensive contextual cues for enhanced reconstruction accuracy and consistency.

The context representation is realized through a set of lightweight neural sub-networks that quickly adapt at test time via self-supervised objectives, substantially increasing memory capacity without significant computational overhead.

Results

Experiments on multiple large-scale benchmarks, including KITTI Odometry and the Oxford Spires dataset, show that the method is effective at handling very large scenes, achieving:

Leading pose accuracy
State-of-the-art 3D reconstruction accuracy
Maintained efficiency