Automated White Blood Cell Counting in Nailfold Capillary Using Deep Learning Segmentation and Video Stabilization

Byeonghwi Kim, Yuli Sun Hariyani, Young-Ho Cho, Cheolsoo Park2020 · DOI

TL;DR

This paper introduces an automated system for counting white blood cells (WBCs) in nailfold capillary videos. By combining deep learning-based capillary segmentation with video stabilization, the method achieves more accurate and robust WBC event detection, outperforming traditional approaches and approaching human expert accuracy.

What Problem Does This Paper Address?

Monitoring white blood cell (WBC) counts is crucial for diagnosing and managing various health conditions, from infections to cancer. Traditional methods often involve invasive blood draws, which can be inconvenient and uncomfortable for patients requiring frequent monitoring. Non-invasive optical methods offer a promising alternative, and one such approach leverages the visual gaps created by WBCs as they flow through nailfold capillaries. However, existing methods for analyzing these capillary images often rely on manual or semi-manual segmentation, which is time-consuming, subjective, and hinders scalability. Furthermore, the small movements inherent in capturing these videos can introduce motion artifacts, further complicating accurate analysis. This paper aims to address these limitations by developing an automated, robust, and scalable system for WBC counting from nailfold capillary videos.

Methods

The proposed method is a pipeline consisting of three main components:

  • Deep Learning-Based Capillary Segmentation: The authors employ a U-Net architecture, a popular convolutional neural network for image segmentation, to automatically identify and segment the capillaries within the video frames. This deep learning approach is chosen for its ability to learn complex features and its efficiency, even with limited training data. This replaces the manual or conventional segmentation methods that were bottlenecks in previous work.
  • Video Stabilization: To mitigate motion artifacts caused by slight hand movements during video capture, a video stabilization technique is applied. This method uses an efficient image registration technique to determine translation vectors between frames. Crucially, it stabilizes the capillary labels obtained from the segmentation, rather than the raw images, to improve robustness against noise. The reference frame is periodically updated to keep translation magnitudes small.
  • WBC Event Detection: After segmentation and stabilization, the method constructs a "spatiotemporal map" (ST map) for each capillary, representing intensity variations over time. WBC events (visual gaps) appear as lines in this ST map. To detect these events, the authors utilize the Radon transform to convert the ST map into a polar map, where lines become peaks. Local maxima detection in the polar map identifies potential WBC events. Post-processing steps are included to filter out duplicate detections and ensure accurate event localization.

Datasets

The study utilized a dataset of 1358 capillary images for training and validation of the deep learning segmentation model. For evaluating the entire WBC counting pipeline, three capillary videos were recorded from healthy subjects. Each video was 30 seconds long, captured at 30 frames per second with a resolution of 640x480. The ground truth for WBC event counting was established by four human experts. No new benchmark dataset was introduced; rather, existing video data was used.

Key Insights and Ideas

  • Deep Learning for Robust Segmentation: The U-Net architecture proved highly effective in segmenting capillaries, producing labels comparable to manual segmentation and significantly outperforming conventional methods, especially in lower-quality images. This automation is key to scalability.
  • Stabilizing Labels, Not Just Images: Stabilizing the segmented capillary labels rather than raw video frames is a clever approach. It focuses the stabilization on the region of interest, making it more robust to noise and variations in the background.
  • Spatiotemporal Maps as a Feature Representation: The transformation of capillary intensity data over time into an ST map provides a powerful representation for detecting transient events like WBCs.
  • Radon Transform for Line Detection: Using the Radon transform to convert the line-detection problem in ST maps into a peak-detection problem in polar maps is an elegant mathematical approach for identifying WBC events.

Key Results

  • The proposed method, "S-DNN" (stabilized deep learning segmentation), consistently predicted WBC event numbers close to the median value determined by human experts across multiple capillaries.
  • The deep learning segmentation method achieved 91.11% validation accuracy (mean intersection over union).
  • Experimental results demonstrated that the combination of deep learning segmentation and video stabilization significantly improved the performance of WBC event counting compared to manual or conventional segmentation, both with and without stabilization.
  • Qualitative results (Figures 8 and A1-A12) show that the proposed method accurately captures the timing and number of events, while other methods often miss events or detect them incorrectly. Notably, some methods failed to detect capillaries altogether in certain cases.

Limitations and Future Work

  • Limited to Translation: The video stabilization method primarily addresses translational motion. Rotational or other complex movements are not explicitly handled, although the periodic updating of the reference frame helps mitigate the impact of larger translations.
  • Specific to Nailfold Capillaries: The method is tailored for nailfold capillary images, and its direct applicability to other types of capillary imaging might require adaptation.
  • Future Work: The authors suggest that further research could explore more advanced video stabilization techniques to handle a wider range of motion artifacts. Additionally, exploring other deep learning architectures or event detection algorithms could potentially further enhance performance.

Summary updated

BibTeX
@misc{kim2020automated,
  author = {Byeonghwi Kim and Yuli Sun Hariyani and Young-Ho Cho and Cheolsoo Park},
  title = {{Automated White Blood Cell Counting in Nailfold Capillary Using Deep Learning Segmentation and Video Stabilization}},
  year = {2020},
  doi = {10.3390/s20247101},
}

Loading related content...

Figure