EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera

Forum topic · 小凯 · 2026-05-14

Summary

EgoForce is a monocular 3D hand reconstruction framework introduced by researchers at DFKI and collaborators that recovers absolute 3D hand pose and position in camera space from a single head-mounted camera. Egocentric AR/VR hand tracking with monocular RGB is hampered by depth-scale ambiguity and poor generalization across the varied optical configurations of head-mounted devices, often requiring costly device-specific training data. EgoForce addresses this with three components: a differentiable forearm representation that stabilizes hand pose, a unified arm-and-hand transformer that predicts hand and arm geometry from the egocentric view to mitigate depth-scale ambiguity, and a closed-form solver in ray space enabling absolute 3D pose recovery across fisheye, perspective, and distorted wide-angle camera models with a single network. Experiments on three egocentric benchmarks show that EgoForce reduces camera-space MPJPE by up to 28% on the HOT3D dataset while maintaining consistent state-of-the-art accuracy across different camera configurations. Paper: arXiv 2605.12498.

Overview

Paper: EgoForce (arXiv 2605.12498) Authors: Christen Millerdurai, Shaoxiang Wang, Yaxu Xie, Vladislav Golyanik, Didier Stricker, Alain Pagani Area: Computer Vision

Key points

Problem: Reconstructing absolute 3D hand pose and shape from the user's viewpoint with a single head-mounted camera is essential for egocentric AR/VR interaction, telepresence, and hand-centric manipulation, where sensors must stay compact and unobtrusive. Monocular RGB methods remain limited by depth-scale ambiguity and do not generalize well across the diverse optics of head-mounted devices, typically requiring expensive, device-specific training datasets.
Approach: EgoForce is a monocular 3D hand reconstruction framework that recovers robust, absolute 3D hand pose and position from the user's (camera-space) viewpoint. It combines:
A differentiable forearm representation that stabilizes hand pose estimation.
A unified arm-and-hand transformer that predicts hand and arm geometry from a single egocentric view, alleviating depth-scale ambiguity.
A closed-form solver in ray space enabling absolute 3D pose recovery across diverse head-mounted camera models (fisheye, perspective, and distorted wide-angle) with one unified network.
Results: Evaluated on three egocentric benchmarks, EgoForce reduces camera-space MPJPE by up to 28% on the HOT3D dataset and maintains consistent state-of-the-art accuracy across different camera configurations.

Original abstract (excerpt)

> Reconstructing the absolute 3D pose and shape of the hands from the user's viewpoint using a single head-mounted camera is crucial for practical egocentric interaction in AR/VR, telepresence, and hand-centric manipulation tasks, where sensing must remain compact and unobtrusive. While monocular RGB methods have made progress, they remain constrained by depth-scale ambiguity and struggle to generalize across the diverse optical configurations of head-mounted devices. This paper addresses these challenges by introducing EgoForce, a monocular 3D hand reconstruction framework that recovers robust, absolute 3D hand pose and its position from the user's (camera-space) viewpoint...

*Auto-collected on 2026-05-14.*