InfiniHand: Streaming World-Space Hand Motion Estimation from Egocentric Video

Forum topic · 小凯 · 2026-09-30

Summary

InfiniHand (arXiv: 2609.35743) is an end-to-end streaming feed-forward framework that estimates world-space hand motion from uncalibrated egocentric video. Unlike prior approaches that cascade independent hand pose estimators with SLAM systems—leading to error accumulation and heavy computational overhead—InfiniHand jointly estimates MANO hand parameters, camera trajectories, and hand locations in a unified architecture. It combines persistent spatiotemporal memory with hand-centered visual features to explicitly couple camera egomotion with local hand geometry. The model is trained in two progressive stages: first learning robust camera-space hand priors, then extending to streaming world-space reconstruction, using roughly 5,000 hours of first-person data aggregated from multiple public datasets. On in-domain benchmarks, InfiniHand outperforms state-of-the-art baselines, reducing ARCTIC PA-p by 21.4% compared to ViDiHand while significantly mitigating world-space drift, and generalizes robustly to in-the-wild videos at 11.19 FPS—more than twice the throughput of HaWoR.

论文概要

Research area: Computer Vision Authors: Kerui Ren, Kaiwen Song, Weiguang Zhao, Yuxi Wang, Yufei Liu, Bo Dai, Haoyu Guo, Chunhua Shen, Mulin Yu, Tao Lu, Junting Dong Published: 2026-09-28 arXiv: 2609.35743

Overview

World-space hand motion estimation from egocentric video requires recovering 3D articulated hand geometry while tracking camera egomotion. Existing approaches heavily rely on cascading independent hand pose estimators and SLAM systems, resulting in error accumulation, complex pipelines, and severe computational overhead. To address these limitations, the authors present InfiniHand, an end-to-end streaming feed-forward framework that jointly estimates MANO parameters, camera trajectories, and hand locations directly from uncalibrated egocentric video.

InfiniHand integrates persistent spatiotemporal memory with hand-centered visual features, explicitly coupling camera motion with local hand geometry within a unified architecture.

Method

Unified architecture: A single feed-forward model replaces cascaded hand pose estimation plus SLAM pipelines.
Persistent spatiotemporal memory: Maintains long-term context to couple camera motion with local hand geometry.
Two-stage progressive training: First learns robust camera-space hand priors, then extends to streaming world-space reconstruction.
Training data: Approximately 5,000 hours of first-person video aggregated from multiple public datasets.

Results

Outperforms state-of-the-art baselines on in-domain benchmarks.
Reduces ARCTIC PA-p by 21.4% compared to ViDiHand, while significantly mitigating world-space drift.
Robustly generalizes to in-the-wild videos.
Runs at 11.19 FPS, more than 2× the throughput of HaWoR.