A new, feasible, and convenient method based on semantic segmentation and deep learning for hemoglobin monitoring

Xiao Hu, Yujie Li, Xin Shu, Ai-lin Song, Hao Liang, Yi-zhu Sun, Xianfeng Wu, Yong-shuai Li, Li-fang Tan, Zhiyong Yang, Chunyong Yang, Linquan Xu, Yu‐Wen Chen, Yi BinFrontiers in Medicine 2023 · DOI

Abstract

We developed a new and automatic method for Hb monitoring to help medical staffs' decision-making with high efficiency, especially in cases of disaster rescue, casualty transport, and so on.

Introduction

Hemoglobin (Hb) monitoring is a cornerstone of clinical decision-making, particularly for blood transfusions and managing anemia. Currently, the "gold standard" involves invasive blood draws (arterial blood gas or venous analysis), which require specialized equipment, trained personnel, and significant time.

While non-invasive alternatives like pulse co-oximetry (SpHb) exist, they are often expensive, hardware-dependent, and sensitive to physiological factors like blood flow and temperature. Previous attempts to use computer vision for Hb estimation typically required manual intervention to "crop" the palpebral conjunctiva (the pink lining of the lower eyelid) from a photo. This study addresses the "automation gap" by creating a system that can autonomously identify the region of interest and estimate Hb levels using a standard smartphone.

Methods

The researchers developed a two-stage deep learning architecture designed for mobile deployment:

  1. Semantic Segmentation (Mask R-CNN): This stage automatically detects and masks the palpebral conjunctiva from a standard "eye-and-face" photo. By performing pixel-level segmentation, the model ensures that subsequent analysis is restricted to the relevant tissue, eliminating the need for manual cropping.
  2. Regression (MobileNetV3): The segmented image is processed by MobileNetV3, a lightweight convolutional neural network (CNN). The network's architecture was modified to output a continuous Hb value (g/dL) rather than a classification label.
Flow diagram for estimating Hb concentration based on different algorithms and inputs.

The study compared this automated pipeline against traditional feature engineering (using color and texture features) and models using manually selected conjunctiva images to validate the effectiveness of the automated approach.

Dataset and Preprocessing

The model was trained and validated using a dataset of 1,065 images from 284 surgical patients. Photos were captured using a smartphone (20 MP) under routine operating room lighting. Ground truth Hb values were obtained via arterial blood gas (ABG) analysis within 10 minutes of image capture. To ensure data quality, images were normalized and K-means clustering was applied to neutralize specular reflections. The final dataset, after up-sampling, consisted of 1,226 training images and 288 test images.

Key Results

The automated Mask R-CNN and MobileNetV3 pipeline demonstrated performance comparable to manual expert-driven methods.

  • Accuracy and Correlation: The model achieved a correlation coefficient ($r$) of 0.77, indicating a strong positive relationship between estimated and actual Hb levels. The Mean Absolute Error (MAE) was 1.6 g/dL.
  • Anemia Screening: When evaluated as a screening tool for anemia (Hb < 10.0 g/dL), the system achieved an accuracy of 85.4% and a high specificity of 97.2%, with an AUROC of 0.752.
Scatter plot showing the correlation between actual and estimated Hb levels.

Discussion and Insights

A significant finding of this study is that automated segmentation performs virtually identically to manual selection ($R^2$ of 0.503 vs. 0.509). This proves that human intervention is no longer a bottleneck for vision-based Hb monitoring. Furthermore, the deep learning approach significantly outperformed traditional hand-crafted features, suggesting that the CNN identifies subtle spatial and spectral patterns that simple color ratios miss.

By utilizing the lightweight MobileNetV3 architecture, the researchers demonstrated that this complex analysis is computationally efficient enough to run locally on mobile devices. This makes the technology highly feasible for field rescue, disaster scenarios, and resource-limited settings where traditional lab equipment is unavailable.

Limitations

While promising, the study notes several areas for improvement. The dataset had a limited number of severe anemia cases, which may affect precision at very low Hb levels. Additionally, as a single-center study, further multicenter validation is required to ensure the model's robustness across different skin tones, lighting conditions, and camera hardware. Future work will also focus on addressing potential labeling biases between different blood analysis methods.

Summary updated

Abstract

We developed a new and automatic method for Hb monitoring to help medical staffs' decision-making with high efficiency, especially in cases of disaster rescue, casualty transport, and so on.
BibTeX
@inproceedings{hu2023new,
  author = {Xiao Hu and Yujie Li and Xin Shu and Ai-lin Song and Hao Liang and Yi-zhu Sun and Xianfeng Wu and Yong-shuai Li and Li-fang Tan and Zhiyong Yang and Chunyong Yang and Linquan Xu and Yu‐Wen Chen and Yi Bin},
  title = {{A new, feasible, and convenient method based on semantic segmentation and deep learning for hemoglobin monitoring}},
  booktitle = {Frontiers in Medicine},
  year = {2023},
  doi = {10.3389/fmed.2023.1151996},
}
Figure