Remote Blood Pressure Estimation via the Spatiotemporal Mapping of Facial Videos

Yuheng Chen, Jialiang Zhuang, Bin Li, Yun Zhang, Xiujuan Zheng2023 · DOI

TL;DR

This paper introduces BPE-Net, an end-to-end deep learning model that estimates blood pressure (BP) from facial videos. By leveraging spatiotemporal feature maps and a novel classification-regression approach, it achieves promising accuracy for non-contact BP monitoring, addressing the inconvenience of traditional methods.

What Problem Does This Paper Address?

Accurate and convenient blood pressure (BP) monitoring is crucial for managing cardiovascular diseases. Traditional BP measurement methods, like using cuffs, are contact-based, leading to inconvenience and discomfort, especially for long-term monitoring or in specific patient populations (e.g., bedridden, burned). While remote photoplethysmography (rPPG) offers a non-contact alternative, it faces challenges like motion artifacts, ambient light interference, and the need for multiple cameras. Existing rPPG-based BP estimation methods often struggle with accuracy and require complex setups. This paper aims to develop an efficient, end-to-end network for remote BP estimation using only facial videos, making daily BP monitoring more accessible and user-friendly.

Methods

The proposed BPE-Net employs a multi-stage approach:

  • Spatiotemporal Feature Map Slices (STSs): To capture subtle blood volume pulse (BVP) signals from facial videos, the method first extracts spatiotemporal maps. This involves:
  • ROI Definition: Identifying four key regions of interest (forehead and cheek) known for rich vascular information.
  • Initial Spatiotemporal Map: Calculating the average pixel values within these ROIs across frames to represent physiological information.
  • Data Augmentation: Applying a modified YUV color space transformation and random masking to enhance robustness against illumination variations and motion artifacts.
  • Spatiotemporal Slicer: Processing these enhanced maps to generate stable pulse wave feature maps by slicing them into manageable segments (STSs) using a sliding window approach.
  • Blood Pressure Estimation Network (BPE-Net):
    • Feature Extraction: A backbone network (ResNet18) is used to extract high-dimensional features from the STSs, followed by a Bi-directional Long Short-Term Memory (BiLSTM) layer to capture temporal correlations.
  • Joint Classification and Regression: A novel approach is introduced where a BP classifier first categorizes the BP into ranges, and a BP calculator then estimates the specific value within that range. This transforms the regression problem into a joint classification-regression task, aiming to improve accuracy and reduce overfitting.
  • Loss Function: A combined loss function is used, comprising cross-entropy loss for the classifier and L1 loss for the calculator. L1 loss is chosen for its robustness to extreme values, which are common in BP data.
  • Oversampling Training Strategy: To address the issue of imbalanced BP distributions in datasets, an oversampling strategy is employed. Data is divided into four groups based on BP values, and samples are drawn from each group in a balanced ratio (1:1:1:1) for each training batch.

Datasets

  • MMSE-HR [31]: A public, non-contact heart rate and blood pressure estimation dataset. It contains 102 facial videos from 40 subjects, recorded at 25 fps. This dataset was used for validation.
  • MPM-BP (Private Dataset): A newly collected, multimodal dataset for non-contact BP estimation. It includes physiological data and facial videos of 132 individuals (aged 18-24), recorded at 30 fps with a resolution of 1920x1080. This dataset was used for training and testing.

Key Insights and Ideas

  • Spatiotemporal Maps for BP: The paper demonstrates that spatiotemporal maps derived from facial videos contain sufficient hidden blood-flow information for BP estimation, moving beyond simple rPPG signal extraction.
  • Joint Classification-Regression: Transforming the BP estimation into a joint classification-regression problem is a novel approach. The insight is that by first classifying BP into ranges, the subsequent regression can be more focused and accurate, especially given the inherent difficulty in predicting precise values.
  • Modified YUV Color Space: The use of a modified YUV color space, which emphasizes the brightness channel, proves more effective than traditional RGB or YUV spaces for capturing subtle BVP signals, likely by better visualizing the intensity of heart rhythm in skin regions.
  • Oversampling for Imbalance: The oversampling strategy is crucial for handling the skewed distribution of BP values in datasets, ensuring that the model learns effectively across all BP ranges, not just the most frequent ones.
  • Optimal Slice Length: The study highlights that the length of the spatiotemporal slices significantly impacts performance, with 150 frames found to be optimal for balancing the capture of physiological information and avoiding excessive non-physiological noise.

Key Results

  • Performance on MPM-BP: The proposed BPE-Net achieved MAE of 12.35 mmHg and RMSE of 16.55 mmHg for systolic BP (SBP), and MAE of 9.54 mmHg and RMSE of 12.22 mmHg for diastolic BP (DBP).
  • Color Space Comparison: The modified YUV color space outperformed RGB and standard YUV, reducing MAE for SBP from 8.42 mmHg (RGB) to 8.07 mmHg (modified YUV).
  • Spatiotemporal Slicer Impact: Using a 150-frame slice length resulted in the best performance, with MAEs of 8.07 mmHg (SBP) and 6.78 mmHg (DBP). This was significantly better than methods without slicing.
  • Oversampling Benefit: The oversampling strategy reduced MAE for SBP from 8.27 mmHg to 8.07 mmHg and for DBP from 6.92 mmHg to 6.78 mmHg.
  • Cross-Dataset Testing (MMSE-HR): The BPE-Net (with 150-slice length) achieved the best performance among compared methods, with an MAE of 12.35 mmHg for SBP and 9.54 mmHg for DBP, outperforming NCBP and NIBPP.

Limitations and Future Work

  • Dataset Limitations: The MPM-BP dataset, while new, has a limited age range (18-24) and may not fully represent diverse demographics. The distribution of BP values was also unbalanced, which the oversampling strategy helped mitigate but doesn't entirely solve.
  • Environmental Applicability: All experiments were conducted indoors. The paper acknowledges that the method's applicability in different environments (varying lighting, ages, or with makeup) needs further verification.
  • Accuracy Standards: The achieved accuracy (e.g., <10 mmHg error probability of 74.2% for SBP and 78.8% for DBP) did not fully meet the stringent requirements of BHS and AAMI standards (requiring >85% probability).
  • Future Work: The authors suggest expanding the dataset diversity to include more age groups and skin conditions. They also propose improving the classification ability for different BP intervals and further verifying the method's performance in diverse real-world scenarios.

Summary updated

BibTeX
@misc{chen2023remote,
  author = {Yuheng Chen and Jialiang Zhuang and Bin Li and Yun Zhang and Xiujuan Zheng},
  title = {{Remote Blood Pressure Estimation via the Spatiotemporal Mapping of Facial Videos}},
  year = {2023},
  doi = {10.3390/s23062963},
}
Figure