Abstract
Precise measurement of physiological signals is critical for effective vital sign monitoring. Recent computer vision advances allow extracting pulse and respiration rates from digital video, enabling contactless monitoring. This paper introduces "StressNet," a novel approach for obtaining physiological signals and classifying stress states from thermal video. StressNet utilizes a hybrid emission representation model to capture heat emission and absorption by skin and blood vessels. This rich feature set enables a spatio-temporal network to reconstruct the Initial Systolic Time Interval (ISTI), a quantitative index of cardiac sympathetic activity and stress. The reconstructed ISTI signal is then used to classify stress states. Evaluation shows StressNet estimates ISTI with 95% accuracy and detects stress with an average precision of 0.842.
Introduction: The Challenge of Remote Stress Monitoring
Remote health monitoring has progressed significantly, yet most contactless systems rely on RGB video to estimate heart rate (HR) or heart rate variability (HRV). These metrics are imperfect proxies for stress as they are influenced by both the sympathetic and parasympathetic nervous systems.
This research targets the Initial Systolic Time Interval (ISTI). ISTI measures the time difference between the heart's electrical activation (ECG R-peak) and the mechanical opening of the aortic valve (B-point in impedance cardiography). ISTI serves as a "gold standard" for quantifying acute stress because it is a pure index of cardiac sympathetic activity. Traditionally, measuring ISTI requires bulky, wired sensors (ECG and ICG). StressNet aims to estimate this signal non-invasively using only thermal video.
Methods: The StressNet Architecture
StressNet is a deep learning framework designed to process thermal video and output a stress probability. The architecture consists of four primary components:
- Emission Representation Model: Unlike models for RGB video that rely on reflected light, this module models direct heat emission and absorption. It employs a log non-linearity to isolate subtle physiological pulses from static facial heat.
- Spatial Module: A ResNet-50 backbone, pre-trained on ImageNet, extracts high-level features from thermal frames. Average pooling is utilized to preserve subtle signals that might be lost with max pooling.
- Temporal Module: A 6-layer LSTM network processes spatial features in 1-second intervals to match the human cardiac cycle, capturing the relationship between consecutive heartbeats.
- Detection & Classification: The model uses a binned approach to predict ISTI values for stability. The resulting signal is fed into a classifier to determine the probability of stress.
Experimental Setup and Dataset
The researchers developed a specialized dataset from the Biomarkers of Stress States (BOSS) study involving 42 healthy adults. Stress was induced using the Cold Pressor Test (CPT), involving foot immersion in $0.5^\circ C$ water, with a Warm Pressor Test ($34^\circ C$) as a control. Ground truth ISTI values were calculated from simultaneous high-fidelity ECG and ICG recordings. High-resolution thermal video was captured at 15 Hz.
Key Results and Insights
StressNet demonstrates that predicted ISTI is a superior predictor of stress compared to traditional metrics.
- Signal Accuracy: The model achieved a Pearson Correlation Coefficient of 0.843 and a Mean Squared Error (MSE) of 5.845 for ISTI prediction, outperforming existing methods like DeepPhys.
- Stress Detection: The system achieved an Average Precision (AP) of 0.842 in classifying stress vs. no-stress states. Notably, ground truth HR and HRV only reached 0.753 AP and 0.814 AP, respectively, validating ISTI as a more robust metric.
- Physics-Informed Preprocessing: Ablation studies showed that removing the Emission Representation step increased MSE by 1.119, highlighting the importance of modeling thermal physics.
Discussion and Future Work
While StressNet offers a privacy-preserving (thermal imaging does not store a "true likeness") and effective means of monitoring stress, limitations remain. The model is sensitive to large head movements and facial obstructions. Future research will explore generalization to social or mental stress, larger datasets to reduce reliance on pre-trained backbones, and performance in less controlled, real-world environments.
Summary updated