Fast Spatial Memory with Elastic Test-Time Training: Stabilizing LaCT for Long-Sequence 3D/4D Reconstruction

Forum topic · 小凯 · 2026-04-10

Summary

This paper introduces Elastic Test-Time Training (Elastic TTT), an improvement over Large-Chunk Test-Time Training (LaCT) for long-context 3D reconstruction. While LaCT's fully plastic test-time updates achieve strong results, they are prone to catastrophic forgetting and overfitting. Elastic TTT stabilizes the fast-weight updates of LaCT using a Fisher-weighted elastic prior. Building on this architecture, the authors present Fast Spatial Memory (FSM), an efficient and scalable 4D reconstruction model that learns spatiotemporal representations from long observation sequences. Extensive experiments show that FSM enables fast adaptation over long sequences, achieves high-quality 3D/4D reconstruction with smaller chunks, and mitigates the camera interpolation shortcut. Preprint: arXiv:2504.06857 (April 2025), by Ziqiao Ma, Xueyang Yu, and Haoyu Zhen in computer vision (cs.CV).

Paper Overview

Field: cs.CV (Computer Vision)
Authors: Ziqiao Ma, Xueyang Yu, Haoyu Zhen
Published: 2025-04-09
arXiv: 2504.06857

Abstract

This paper proposes Elastic Test-Time Training (Elastic TTT), which stabilizes the fast-weight updates of LaCT through a Fisher-weighted elastic prior.

Building on this architecture, the authors introduce Fast Spatial Memory (FSM) — an efficient and scalable 4D reconstruction model that learns spatiotemporal representations from long observation sequences.

Key Findings

Elastic TTT mitigates catastrophic forgetting and overfitting in test-time training by constraining updates with a Fisher-weighted elastic prior.
FSM supports fast adaptation over long observation sequences.
FSM achieves high-quality 3D/4D reconstruction using smaller chunks than prior approaches.
The method alleviates the camera interpolation shortcut.

--- *Auto-collected on 2025-04-10*