Paper Overview
Field: Computer Vision Authors: Zunnan Xu, Shangchen Zhou, Yihang Luo, Chen Change Loy Published: 2026-10-22 arXiv: 2610.31635
Abstract
Feed-forward models accurately estimate camera motion and 3D geometry from short video windows. However, computational and memory costs prevent direct inference on sequences containing thousands of frames. Long videos must therefore be processed as overlapping chunks. This division produces independent local reconstructions with inconsistent coordinate frames and duplicate estimates of shared cameras.
Achieving global consistency requires propagating constraints from adjacent overlaps and distant revisits across the sequence. This requirement naturally motivates pose graph optimization. However, the choice of graph level creates a trade-off: chunk-level graphs efficiently resolve scale drift and close large loops, yet their rigid transformations cannot correct internal errors or conflicting estimates within chunks.
HiP3R addresses this trade-off with a hierarchical pose graph:
1. Chunk-level Sim(3) optimization — first builds a robust chunk-level Sim(3) graph to establish global layout and scale. 2. Per-frame SE(3) estimation — with the layout fixed, it estimates a shared SE(3) camera for each frame, converting overlapping-window predictions into a single unified trajectory.
Experiments on multiple benchmarks show that HiP3R effectively improves camera trajectory quality and 3D reconstruction. Notably, it applies to both chunk-based and streaming backends without additional training or fine-tuning.
---
*Auto-collected on 2026-10-26*