UR-Net: An Integrated ResUNet and Attention Based Image Enhancement and Classification Network for Stain-Free White Blood Cells

Sikai Zheng, Xiwei Huang, Jin Chen, Zefei Lyu, Jingwen Zheng, Jiye Huang, Haijun Gao, Shan Liu, Lingling Sun2023 · DOI

TL;DR

This paper introduces UR-Net, a novel end-to-end deep learning architecture that jointly enhances stain-free white blood cell (WBC) images and classifies them. By integrating a ResUNet-based image enhancement module with a ResNet101 classifier and incorporating attention mechanisms, UR-Net achieves a state-of-the-art classification accuracy of 83.34% on stain-free WBC images, overcoming challenges posed by their low contrast and lack of staining.

What Problem Does This Paper Address?

Accurate white blood cell (WBC) classification is crucial for diagnosing various diseases. Traditional methods rely on manual examination of stained blood smears, which is time-consuming, requires specialized reagents, and can alter cell morphology. While stain-free WBC imaging offers advantages, these images often suffer from low quality due to illumination variations and the inherent low contrast of unstained cell nuclei, making classification difficult for automated systems. Existing image enhancement techniques, while improving visual quality, don't always translate to better performance in downstream computer vision tasks like classification. This paper addresses the need for an integrated approach that enhances stain-free WBC images specifically to improve classification accuracy.

Methods

The core of this work is the UR-Net architecture, which combines an image enhancement module and a classification module into a single, end-to-end trainable network.

  • Image Enhancement Module (Modified ResUNet): The authors adapt the ResUNet architecture for image enhancement. Key modifications include:
  • Using a 3x3 convolution kernel with padding of 1 and stride of 1 to maintain feature map size and facilitate residual connections.
  • Replacing pooling layers with convolutional layers to prevent loss of potentially crucial features in low-contrast stain-free images.
  • Employing LeakyReLU as the activation function to mitigate the "dying ReLU" problem.
  • Introducing a 1x1 convolutional layer in cross-layer connections to improve feature fusion between shallow and deep layers.
  • Integrating channel attention mechanisms (squeeze-and-excitation) to adaptively highlight important features and suppress background noise, directing the network's focus towards relevant cell structures.
  • Classification Module (ResNet101): A pre-trained ResNet101 model is used for its proven classification capabilities.
  • Joint Training and Pre-training: The enhancement and classification modules are integrated and trained jointly. Crucially, the enhancement module is pre-trained by setting the input and output images to be the same. This process helps initialize the weights to a favorable state, leading to faster convergence and improved accuracy during fine-tuning on the WBC dataset.

Datasets

The study utilizes a stain-free WBC dataset previously collected from healthy donors.

  • Data Collection: Blood samples were processed using a microfluidic chip to isolate WBCs. These were then fluorescently stained for visualization, and both brightfield and fluorescence images were captured.
  • Segmentation: Brightfield images were segmented into 200x200 pixel images.
  • Augmentation: To address data imbalance and mitigate overfitting, rotation and flipping techniques were applied, augmenting the dataset to approximately 10,000 images.
  • Splitting: The dataset was split into training (80%) and testing (20%) subsets.

Key Insights and Ideas

  • Enhancement for Classification, Not Just Perception: A critical insight is that image enhancement techniques should be optimized for downstream tasks (like classification) rather than solely for human visual appeal. UR-Net's joint training approach embodies this principle.
  • Attention is Key for Stain-Free Images: The incorporation of channel attention mechanisms proved vital. It allowed the model to effectively focus on the subtle features of WBC nuclei while ignoring distracting background noise, which is a significant challenge in stain-free imaging.
  • Pre-training Accelerates Convergence and Boosts Accuracy: Pre-training the enhancement module with an identity mapping (input=output) provided a strong initialization, enabling faster convergence and higher final accuracy with fewer training epochs. This is particularly beneficial for complex, integrated models.
  • Integrated Architectures Outperform Modular Approaches: The experimental results strongly suggest that an end-to-end integrated architecture (UR-Net) performs better than separate enhancement and classification modules, even when those modules are individually strong.

Key Results

  • Optimal Accuracy: UR-Net achieved a best classification accuracy of 83.34% on the stain-free WBC dataset.
  • Comparison to Baselines:
    • UR-Net outperformed a single ResNet101 classifier (Net1) by a significant margin.
  • It also surpassed models with a non-modified ResUNet enhancement module (Net2) and a ResUNet enhancement module without attention (Net3).
  • The pre-trained UR-Net (Net4) showed faster convergence and higher accuracy compared to models trained from scratch.
  • Qualitative Enhancement: Figure 3a demonstrates that UR-Net significantly improves image contrast and sharpens edge details in stain-free WBC images, making cellular structures more discernible.
  • Attention Mechanism Effectiveness: Grad CAM visualizations (Figure 3b) clearly show that the attention mechanism helps the model focus on cell features rather than background noise, leading to improved classification.
  • Monocyte Recall Improvement: Despite the inherent challenge of imbalanced monocyte data, UR-Net achieved a nearly 9% increase in monocyte recall after pre-training, highlighting the effectiveness of the enhanced feature extraction.

Limitations and Future Work

  • Environmental Robustness: The authors acknowledge that the model's performance has not been tested on WBC images acquired under diverse or adverse environmental conditions, and its stability in such scenarios remains to be investigated.
  • Future Directions:
    • Refine experiments and further optimize the network for stain-free WBC classification.
  • Investigate the model's applicability to other high-level computer vision tasks, such as object detection.
  • Explore testing on WBC images from different environments to assess robustness.

Summary updated

BibTeX
@misc{zheng2023ur,
  author = {Sikai Zheng and Xiwei Huang and Jin Chen and Zefei Lyu and Jingwen Zheng and Jiye Huang and Haijun Gao and Shan Liu and Lingling Sun},
  title = {{UR-Net: An Integrated ResUNet and Attention Based Image Enhancement and Classification Network for Stain-Free White Blood Cells}},
  year = {2023},
  doi = {10.3390/s23177605},
}
Figure