E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation

Forum topic · 小凯 · 2026-04-12

Summary

E-3DPSM is an event-driven continuous pose state machine for egocentric 3D human pose estimation using event cameras on head-mounted devices. Event cameras offer millisecond temporal resolution, high dynamic range, and negligible motion blur, but prior methods suffer from low 3D accuracy, self-occlusion sensitivity, and temporal jitter because their designs do not fully match the asynchronous, continuous nature of event streams. E-3DPSM addresses this by aligning continuous human motion with fine-grained event dynamics: it evolves a latent state and predicts continuous changes in 3D joint positions tied to observed events, which are fused with direct 3D pose prediction to yield stable, drift-free reconstructions. The system runs in real time at 80 Hz on a single workstation and sets new records on two benchmarks, improving accuracy (MPJPE) by up to 19% and temporal stability by up to 2.7x. Paper by Mayur Deshmukh, Hiroyasu Akada, and Helge Rhodin, arXiv:2504.07880.

Paper Overview

Field: Computer Vision Authors: Mayur Deshmukh, Hiroyasu Akada, Helge Rhodin Published: 2025-04-10 arXiv: 2504.07880

Abstract

Event cameras offer multiple advantages in monocular egocentric 3D human pose estimation from head-mounted devices, such as millisecond temporal resolution, high dynamic range, and negligible motion blur. Existing methods effectively leverage these properties, but suffer from low 3D estimation accuracy, insufficient in many applications (e.g., immersive VR/AR). This is due to the design not being fully tailored towards event streams (e.g., their asynchronous and continuous nature), leading to high sensitivity to self-occlusions and temporal jitter in the estimates.

This paper rethinks the setting and introduces E-3DPSM, an event-driven continuous pose state machine for event-based egocentric 3D human pose estimation. E-3DPSM aligns continuous human motion with fine-grained event dynamics; it evolves a latent state and predicts continuous changes in 3D joint positions associated with observed events, which are fused with direct 3D human pose prediction, resulting in stable and drift-free final 3D pose reconstruction.

Key Results

Runs in real time at 80 Hz on a single workstation
New state-of-the-art on two benchmarks
Accuracy (MPJPE) improved by up to 19%
Temporal stability improved by up to 2.7x