Pretrained Vision Models for Predicting High-Risk Breast Cancer Stage

Bonaventure F. P. Dossou, Yenoukoume S. K. Gbenou, Miglanche Ghomsi Nono

TL;DR

This paper explores using pre-trained computer vision models to predict breast cancer stage from digital pathology (biopsy) images. The authors found that an ensemble of these models significantly outperforms individual models, offering a winning solution for the Nightingale Breast Cancer challenge.

What Problem Does This Paper Address?

Breast cancer is a leading cause of death globally, and early detection is crucial for effective treatment. While mammography and MRI are common screening tools, breast biopsies provide detailed cellular information that can help determine the cancer's stage. However, manually analyzing these biopsy slides is time-consuming and requires expert knowledge. This paper addresses the challenge of automating breast cancer staging using digital pathology images, aiming to improve efficiency and accuracy by leveraging the power of pre-trained deep learning models.

Methods

The core of this work involves applying pre-trained Convolutional Neural Networks (CNNs) to predict breast cancer stages from digital biopsy images. The authors experimented with several state-of-the-art vision models, including ResNet, EfficientNet, ConvNeXt, WideResNet, VGG, RegNet, Swin Transformer, and MaxViT.

Key methodological aspects include:

  • Transfer Learning: Utilizing models pre-trained on large image datasets (like ImageNet) and fine-tuning them on the specific task of breast cancer staging. This leverages learned features from general image recognition to improve performance on medical images.
  • Image Preprocessing: Downsampling high-resolution biopsy images to a uniform size (224x224 pixels) compatible with most pre-trained models. Data augmentation techniques like random cropping and horizontal flipping were applied to the training set to enhance robustness.
  • Ensemble Learning: A crucial finding is that combining the predictions of multiple individual models (an ensemble) leads to superior performance compared to any single model. They explored two ensemble strategies: one using all models and another using only models with a Mean Square Error (MSE) below 1, which yielded the best results.
  • Stage Prediction: For each biopsy, the final stage prediction is an average of the predictions from its individual slide images.

Datasets

The primary dataset used in this study is the Nightingale Open Science Dataset of Digital Pathology (NOSDDP). This dataset contains:

  • 72,400 biopsy slide images from 4,335 breast biopsies of 3,425 unique patients.
  • Data collected between 2014 and 2020.
  • Crucially, these images are linked to patient cancer stages and mortality information.

The dataset was split into training, validation, and testing sets. A notable aspect is the presence of biopsies without assigned stages, which the authors handled by assigning them to the most frequent stage (Stage 1) for training purposes.

Key Insights and Ideas

  • Ensembles are King: The most significant insight is that combining multiple pre-trained vision models (deep ensembles) dramatically improves breast cancer staging accuracy over individual models. This mirrors the real-world scenario where aggregating expert opinions leads to better diagnoses.
  • Leveraging Pre-trained Models is Effective: The study demonstrates that even without extensive medical imaging pre-training, general-purpose vision models can be effectively fine-tuned for complex medical tasks like cancer staging.
  • Model Architecture Matters (to an extent): While ensembles are superior, EfficientNet showed promising individual performance, possibly due to its architecture that combines families of neural networks and uses mobile inverted bottleneck convolutions.
  • The Value of Data: The NOSDDP dataset, linking biopsy images to clinical outcomes, is a valuable resource for advancing AI in breast cancer research.

Key Results

  • The deep ensemble model achieved a Mean Square Error (MSE) of 0.5543481, which was declared the "winning solution" for the Nightingale Predicting High-Risk Breast Cancer Contest.
  • This ensemble performance was significantly better than individual models, with the best individual model (EfficientNet) achieving an MSE of 0.829745 (at a specific learning rate).
  • The study also highlights that ensembles using models with lower individual losses tend to have lower overall ensemble MSE.

Limitations and Future Work

  • Interpretability: While the models achieve good performance, the paper acknowledges the need for better interpretability. The authors suggest exploring Causal Inference (CI) as a future direction to understand the causal relationships between image features and predictions, which could lead to more robust and trustworthy models.
  • Uncertainty Estimation: The authors also plan to investigate how uncertainty estimation can be incorporated into medical imaging tasks to provide a measure of confidence in the model's predictions.
  • Handling Unlabeled Data: The strategy of assigning unlabeled biopsies to the most frequent stage is a pragmatic approach but might introduce some bias. Future work could explore more sophisticated methods for handling such data.

Summary updated

Abstract

Cancer is increasingly a global health issue. Seconding cardiovascular diseases, cancers are the second biggest cause of death in the world with millions of people succumbing to the disease every year. According to the World Health Organization (WHO) report, by the end of 2020, more than 7.8 million women have been diagnosed with breast cancer, making it the world's most prevalent cancer. In this paper, using the Nightingale Open Science dataset of digital pathology (breast biopsy) images, we leverage the capabilities of pre-trained computer vision models for the breast cancer stage prediction task. While individual models achieve decent performances, we find out that the predictions of an ensemble model are more efficient, and offer a winning solution\footnote{https://www.nightingalescience.org/updates/hbc1-results}. We also provide analyses of the results and explore pathways for better interpretability and generalization. Our code is open-source at \url{https://github.com/bonaventuredossou/nightingale_winning_solution}
BibTeX
@inproceedings{dossou2023pretrained,
  author = {Bonaventure F. P. Dossou and Yenoukoume S. K. Gbenou and Miglanche Ghomsi Nono},
  title = {{Pretrained Vision Models for Predicting High-Risk Breast Cancer Stage}},
  booktitle = {MLGH @ ICLR},
  year = {2023},
  url = {https://openreview.net/forum?id=pWAJGO_HZe},
}
Figure