Illumination-Robust Camera-Based Heart-Rate Estimation via Spatial-Temporal Transformer (arXiv 2606.12378)

Forum topic · 小凯 · 2026-06-12

Summary

Researchers Zhi Wei Xu and Torbjörn E. M. Nordling propose an end-to-end spatial-temporal transformer framework for non-contact heart-rate estimation from RGB cameras, robust to illumination changes—a key barrier for robot-mounted physiological sensing. The pipeline combines PRNet-based 3D face alignment, clip-level illumination augmentation, a Residual Temporal Standardization Module, and hybrid temporal-frequency supervision. Training uses a Soft-Shifted Pearson waveform loss plus spectral Kullback-Leibler divergence loss, with a tuned weight beta controlling frequency-domain heart-rate guidance. On a new dataset spanning three illumination levels, beta=5 achieved the best results: heart-rate MAE of 0.79 bpm and correlation of 0.982. Compared to a PhysFormer baseline on the same dataset, the estimator reduces heart-rate MAE by 93.6% and lifts correlation from 0.088 to 0.982, demonstrating practical robustness under varying lighting for service, social, and assistive robots. Paper: arXiv 2606.12378.

Paper Overview

Field: Computer Vision (CV)
Authors: Zhi Wei Xu, Torbjörn E. M. Nordling
Published: 2026-06-10
arXiv: 2606.12378

Abstract

Physiological awareness is important for service, social, and assistive robots that interact with humans in everyday environments. Remote photoplethysmography (rPPG) enables non-contact heart-rate (HR) estimation from an RGB camera, making it a promising sensing modality for robot-mounted vision systems. However, illumination variation remains a major barrier to robust deployment.

This paper presents an end-to-end spatial-temporal transformer framework for remote HR estimation on a new dataset with varied illumination. The estimator integrates:

PRNet-based 3D face alignment
Clip-level illumination augmentation
The Residual Temporal Standardization Module
Controlled hybrid temporal-frequency supervision

The training objective combines a Soft-Shifted Pearson waveform loss with a spectral Kullback-Leibler divergence loss, where a tuning weight beta controls the contribution of frequency-domain HR guidance.

Results

In experiments under a full-level mixing protocol covering three illumination levels, beta=5 provided the strongest results among tested settings:

Best running heart-rate MAE: 0.79 bpm
Heart-rate correlation: 0.982

Compared with a PhysFormer baseline evaluated on the same dataset, the proposed estimator:

Reduces heart-rate MAE by 93.6%
Improves heart-rate correlation from 0.088 to 0.982, making it usable under illumination changes

---

*Auto-collected on 2026-06-12.*