EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera

EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera

Field: Computer Vision Authors: Christen Millerdurai, Shaoxiang Wang, Yaxu Xie, Vladislav Golyanik, Didier Stricker, Alain Pagani arXiv: 2605.12498

Key points

• Reconstructing absolute 3D hand pose and shape from the user's viewpoint with a single head-mounted camera is essential for AR/VR, telepresence, and hand-centric manipulation, where sensing must remain compact and unobtrusive.
• Existing monocular RGB methods are limited by depth-scale ambiguity and generalize poorly across the diverse optical configurations of head-mounted devices, typically requiring costly device-specific training datasets.

The EgoForce approach

EgoForce is a monocular 3D hand reconstruction framework that recovers robust, absolute 3D hand pose and position in camera space from the user's viewpoint. Its core components:

• Differentiable forearm representation to stabilize hand pose estimation.
• Unified arm-hand transformer that predicts hand and arm geometry from a single egocentric view, mitigating depth-scale ambiguity.
• Closed-form ray-space solver enabling absolute 3D pose recovery across diverse head-mounted camera models (fisheye, perspective, and distortion wide-angle) with a single unified network.

Results

• Evaluated on three egocentric benchmarks, EgoForce reduces camera-space MPJPE by up to 28% on the HOT3D dataset.
• It maintains consistent state-of-the-art accuracy across different camera configurations, avoiding dependence on device-specific datasets.