Paper Overview
Research Area: Computer Vision Authors: Chin-Yang Lin, Yang-Che Sun, Cheng Sun Published: 2026-09-03 arXiv: 2609.04201
Summary
Online 3D reconstruction models perform poorly on long videos. The root cause is that regressing poses relative to a fixed first-frame anchor forces the model to extrapolate far beyond its training distribution, where tiny drifts continuously accumulate and amplify into severe geometric collapse. However, the authors observe that per-frame depth remains stable throughout this failure process — the backbone's local geometry stays intact, and only the global pose head collapses.
Based on this decoupling observation, the authors propose Scal3R, which reformulates online reconstruction as multi-reference relative pose querying. Lightweight learnable tokens (~1% of parameters) are injected into a fully frozen backbone via asymmetric attention. In this setup, poses are queried relative to multiple past keyframes, and an online pose graph optimization system with loop closure suppresses long-range drift.
Key Results
- Scal3R converges in just 8 hours on a single GPU
- Reduces average ATE by over 60% on KITTI compared to online baselines
- Achieves state-of-the-art performance on Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes
Original Abstract
Online 3D reconstruction models perform poorly on long videos because regressing poses relative to a fixed first-frame anchor forces extrapolation far beyond the training distribution. We introduce Scal3R, which reformulates online reconstruction as multi-reference relative pose querying using lightweight learnable tokens (~1% of parameters) injected into a frozen backbone via asymmetric attention.
--- *Auto-collected on 2026-09-05*