E-3DPSM: An Event-Driven State Machine for Egocentric 3D Human Pose Estimation

Forum topic · 小凯 · 2026-04-12

Summary

E-3DPSM is a new method for monocular egocentric 3D human pose estimation from head-mounted event cameras, presented in arXiv paper 2504.07880 by Mayur Deshmukh, Hiroyasu Akada, and Helge Rhodin. Event cameras offer millisecond temporal resolution, high dynamic range, and negligible motion blur, but existing approaches suffer from low 3D accuracy, self-occlusion sensitivity, and temporal jitter because they are not fully tailored to the asynchronous, continuous nature of event streams. E-3DPSM rethinks this setting as an event-driven continuous pose state machine: it evolves latent states, predicts continuous changes in 3D joint positions aligned with fine-grained event dynamics, and fuses them with direct 3D pose prediction to yield stable, drift-free reconstructions. The system runs in real time at 80 Hz on a single workstation and sets new state-of-the-art records on two benchmarks, improving accuracy (MPJPE) by up to 19% and temporal stability by up to 2.7x, making it practical for immersive VR/AR applications.

Paper Overview

Field: Computer Vision Authors: Mayur Deshmukh, Hiroyasu Akada, Helge Rhodin Published: 2025-04-10 arXiv: 2504.07880

Abstract

Event cameras offer multiple advantages in monocular egocentric 3D human pose estimation from head-mounted devices, such as millisecond temporal resolution, high dynamic range, and negligible motion blur. Existing methods effectively leverage these properties, but suffer from low 3D estimation accuracy, insufficient in many applications (e.g., immersive VR/AR). This is due to the design not being fully tailored towards event streams (e.g., their asynchronous and continuous nature), leading to high sensitivity to self-occlusions and temporal jitter in the estimates.

This paper rethinks the setting and introduces E-3DPSM, an event-driven continuous pose state machine for event-based egocentric 3D human pose estimation. E-3DPSM aligns continuous human motion with fine-grained event dynamics: it evolves latent states and predicts continuous changes in 3D joint positions associated with the observed events. These are fused with direct 3D human pose prediction, producing stable and drift-free final 3D pose reconstruction.

Key Results

Runs in real time at 80 Hz on a single workstation
New state-of-the-art records on two benchmarks
Accuracy improvements of up to 19% (MPJPE)
Temporal stability improvements of up to 2.7x