BarlowTwins-CXR: enhancing chest X-ray abnormality localization in heterogeneous data with cross-domain self-supervised learning

Haoyue Sheng, Linrui Ma, Jean-Francois Samson, Dianbo LiuBMC Medical Informatics Decis. Mak 2024 · OpenReview

TL;DR

This paper introduces BarlowTwins-CXR, a novel self-supervised learning approach that significantly improves the accuracy of detecting and localizing abnormalities in chest X-rays, especially when dealing with data from different sources. By reducing feature redundancy, BarlowTwins-CXR overcomes domain inconsistencies, leading to better performance than traditional methods.

What Problem Does This Paper Address?

Interpreting chest X-rays (CXRs) is crucial for diagnosing many diseases, but it's a complex task that requires highly skilled radiologists. The increasing demand for these services, coupled with a shortage of experts, leads to a growing workload and a higher risk of diagnostic errors. While deep learning has shown promise in automating this process, a major hurdle is "domain inconsistency." This means models trained on one type of CXR data (e.g., from a specific hospital or scanner) often perform poorly on data from a different source due to variations in image characteristics. This paper tackles this challenge by developing a self-supervised learning method to make CXR analysis models more robust and adaptable across different datasets.

Methods

The core of this work is the BarlowTwins-CXR approach, a two-stage training process:

  1. Self-Supervised Pre-training: The researchers adapted the Barlow Twins algorithm for CXR images. Barlow Twins is a self-supervised learning technique that aims to make the features learned by a neural network independent of each other. This is achieved by minimizing the redundancy between features extracted from different augmented views of the same image. The goal here is to learn general, robust visual representations from unlabeled CXR data, specifically from the NIH-CXR dataset. This pre-training refines an existing ResNet50 backbone, which was initially pre-trained on the ImageNet dataset.
  2. Supervised Fine-tuning: The pre-trained BarlowTwins-CXR backbone is then integrated into a Faster R-CNN object detection framework. This framework, known for its effectiveness in precise localization, is fine-tuned on the VinDr-CXR dataset, which contains detailed bounding box annotations for abnormalities. This stage adapts the general features learned during pre-training to the specific task of abnormality detection and localization.

Datasets

The study utilized two publicly available datasets:

  • NIH Chest X-ray Dataset (NIH-CXR): This dataset, containing 112,120 posterior-anterior (PA) or anterior-posterior (AP) CXR images from 30,805 patients, was used for the self-supervised pre-training phase. It includes image-level annotations for 14 diseases and some location annotations.
  • VinDr-CXR Dataset: This is a larger dataset of 18,000 PA CXR scans, specifically curated for object detection. It provides instance-level bounding box annotations for 14 diseases, making it ideal for the supervised fine-tuning stage.

Key Insights and Ideas

  • Self-supervised learning is powerful for medical imaging: The study strongly suggests that self-supervised learning, particularly Barlow Twins, can effectively learn meaningful representations from unlabeled medical images, reducing reliance on costly manual annotations.
  • Reducing feature redundancy is key for cross-domain generalization: By forcing learned features to be independent, Barlow Twins-CXR helps models generalize better to new, unseen data distributions, a critical issue in medical imaging.
  • A two-stage approach is effective: Combining self-supervised pre-training with supervised fine-tuning on a task-specific dataset yields superior results compared to using only standard ImageNet pre-training.
  • Barlow Twins-CXR is more efficient: The method leads to a faster training trajectory and improved detection performance, even with limited data.

Key Results

  • Improved mAP50: The BarlowTwins-CXR approach achieved a 3% increase in mAP50 accuracy compared to traditional ImageNet pre-trained models on the VinDr-CXR dataset.
  • Faster Training: The method demonstrated an "expedited training trajectory" and a significant increase in detection performance.
  • Qualitative Improvements: Ablation CAM (Class Activation Mapping) revealed enhanced precision in localizing chest abnormalities, with heatmaps showing more focused alignment with actual lesion areas.
  • Performance across resolutions: The method showed consistent benefits across different input resolutions (224x224 and 640x640 pixels), although the performance gap narrowed slightly at higher resolutions.
  • Linear Evaluation: In linear evaluation tasks on the NIH-CXR dataset, BarlowTwins-CXR consistently outperformed ImageNet pre-training across various data sizes (1%, 10%, 100%), especially in data-limited scenarios.

Limitations and Future Work

  • Dataset Scarcity: A primary limitation is the scarcity of CXR datasets with bounding box annotations. The reliance on public datasets may limit the generalizability of the findings.
  • Computational Cost: The BarlowTwins pre-training phase is computationally intensive, requiring significant GPU resources and time.
  • Future Work: The authors plan to develop an interactive demo system for real-world deployment, explore more advanced self-supervised learning methods, and investigate object detection frameworks and backbone networks. They also aim to collect proprietary datasets and explore higher image resolutions for pre-training.

Summary updated

Abstract

Chest X-ray imaging based abnormality localization, essential in diagnosing various diseases, faces significant clinical challenges due to complex interpretations and the growing workload of radiologists. While recent advances in deep learning offer promising solutions, there is still a critical issue of domain inconsistency in cross-domain transfer learning, which hampers the efficiency and accuracy of diagnostic processes. This study aims to address the domain inconsistency problem and improve autonomic abnormality localization performance of heterogeneous chest X-ray image analysis, particularly in detecting abnormalities, by developing a self-supervised learning strategy called “BarlwoTwins-CXR”. We utilized two publicly available datasets: the NIH Chest X-ray Dataset and the VinDr-CXR. The BarlowTwins-CXR approach was conducted in a two-stage training process. Initially, self-supervised pre-training was performed using an adjusted Barlow Twins algorithm on the NIH dataset with a Resnet50 backbone pre-trained on ImageNet. This was followed by supervised fine-tuning on the VinDr-CXR dataset using Faster R-CNN with Feature Pyramid Network (FPN). The study employed mean Average Precision (mAP) at an Intersection over Union (IoU) of 50% and Area Under the Curve (AUC) for performance evaluation. Our experiments showed a significant improvement in model performance with BarlowTwins-CXR. The approach achieved a 3% increase in mAP50 accuracy compared to traditional ImageNet pre-trained models. In addition, the Ablation CAM method revealed enhanced precision in localizing chest abnormalities. The study involved 112,120 images from the NIH dataset and 18,000 images from the VinDr-CXR dataset, indicating robust training and testing samples. BarlowTwins-CXR significantly enhances the efficiency and accuracy of chest X-ray image-based abnormality localization, outperforming traditional transfer learning methods and effectively overcoming domain inconsistency in cross-domain scenarios. Our experiment results demonstrate the potential of using self-supervised learning to improve the generalizability of models in medical settings with limited amounts of heterogeneous data. This approach can be instrumental in aiding radiologists, particularly in high-workload environments, offering a promising direction for future AI-driven healthcare solutions.
BibTeX
@inproceedings{sheng2024barlowtwins,
  author = {Haoyue Sheng and Linrui Ma and Jean-Francois Samson and Dianbo Liu},
  title = {{BarlowTwins-CXR: enhancing chest X-ray abnormality localization in heterogeneous data with cross-domain self-supervised learning}},
  booktitle = {BMC Medical Informatics Decis. Mak},
  year = {2024},
  url = {https://openreview.net/forum?id=O1hoJgB3f2},
}
Figure