Remote Photoplethysmography Using Triple-Head Spatio-Temporal Transformer with Reaction-Driven Gating and Illumination Separation

Abstract

Remote Photoplethysmography (rPPG) provides a non-contact alternative to traditional heart rate monitoring. Estimating physiological signals from facial videos has recently attracted significant research interest. However, rPPG performance is sensitive to illumination variation and environmental interference, which can distort the extracted physiological signal. Since the background and face are affected by similar conditions, the effect of these conditions can be extracted from the background and isolated from the result. This paper proposes the Triple-Head Spatio-Temporal Transformer (TH-STT). TH-STT is a multi-task architecture designed to separate rPPG signals from environmental interference. In addition to facial tokens, a background anchor token is used as an environmental reference. Facial tokens and background anchor are processed using a shared transformer backbone. The proposed architecture has two auxiliary tasks to help purify the resulting rPPG. The Reaction-Driven Gating (RDG) mechanism was introduced, which tracks facial muscular activity. Furthermore, a Dynamic Anchor Locking (DAL) strategy is proposed to cancel environmental illumination interference. Experimental results on three benchmark datasets demonstrate improved and stable performance, with the TH-STT achieving a Mean Absolute Error (MAE) of 0.42 bpm on UBFC-rPPG and 1.08 on COHFACE.