Case-level Breast Cancer Prediction for Real Hospital Settings

Shreyasi Pathak, Jörg Schlötterer, Jeroen Geerdink, Jeroen Veltman, Maurice van Keulen, Nicola Strisciuglio, Christin Seifert

Abstract

Breast cancer prediction models for mammography assume that annotations are available for individual images or regions of interest (ROIs), and that there is a fixed number of images per patient. These assumptions do not hold in real hospital settings, where clinicians provide only a final diagnosis for the entire mammography exam (case). Since data in real hospital settings scales with continuous patient intake, while manual annotation efforts do not, we develop a framework for case-level breast cancer prediction that does not require any manual annotation and can be trained with case labels readily available at the hospital. Specifically, we propose a two-level multi-instance learning (MIL) approach at patch and image level for case-level breast cancer prediction and evaluate it on two public and one private dataset. We propose a novel domain-specific MIL pooling observing that breast cancer may or may not occur in both sides, while images of both breasts are taken as a precaution during mammography. We propose a dynamic training procedure for training our MIL framework on a variable number of images per case. We show that our two-level MIL model can be applied in real hospital settings where only case labels, and a variable number of images per case are available, without any loss in performance compared to models trained on image labels. Only trained with weak (case-level) labels, it has the capability to point out in which breast side, mammography view and view region the abnormality lies.

Introduction

Most state-of-the-art mammography models are "label-hungry," typically requiring either Region of Interest (ROI) labels or image-level labels. However, in standard hospital workflows, radiologists provide a single diagnosis for an entire case (the whole exam). This exam usually contains at least four images, but cancer might only be visible in one. Furthermore, clinicians often take extra views to resolve uncertainty, leading to a variable number of images per patient.

Comparison of labeling levels: from ROI-level to the proposed case-level approach for real hospital settings.

This paper addresses the gap between research datasets and clinical reality by developing a framework that predicts breast cancer using only these "weak" case-level labels, handling variable bag sizes without requiring manual annotation of every image.

Methodology

The researchers formulated the problem as a two-level Multi-Instance Learning (MIL) task:

  1. Patch-level MIL: The model uses a GMIC-ResNet18 backbone to break individual images into patches, identifying potential ROIs in an unsupervised manner.
  2. Image-level MIL: These image features are aggregated to make a final case-level prediction.

Key Innovations

  • Domain-Specific Side-wise Pooling: Standard MIL treats all images in a "bag" as equal. This framework introduces a pooling block that pools views for the left and right breasts separately before combining them, reflecting the bilateral nature of mammography.
  • Dynamic Training Procedure: To handle cases with non-standard image counts (e.g., 5 or 6 views), the authors developed a training scheme that groups batches by image combination and resets optimizer states for unused components, preventing gradient history from updating parameters for views not present in a specific batch.

Experimental Results

The model was evaluated on public datasets (CBIS-DDSM, VinDr-Mammo) and a massive private dataset (CLaM) from the Netherlands containing over 21,000 cases. On the CLaM-VV dataset, which captures the true complexity of variable hospital views, the model achieved an AUC of 0.85.

Localization and Robustness

Despite being trained only on weak case-level labels, the model demonstrated impressive localization capabilities. It achieved an Intersection over Union (IoU) of 0.30 on the VinDr dataset for ROI extraction—performance comparable to models explicitly trained with ROI data.

A real-world mammography case with five views (R-CC, R-MLO, R-XCCL, L-CC, L-MLO) showing the model's ability to identify abnormalities across variable views.

Discussion and Conclusion

The most significant insight is that manual annotation is not mandatory for high-performing models. Models trained only on case labels performed as well as those trained on expensive image-level labels. This suggests hospitals can train powerful AI tools using existing diagnostic databases without years of manual labeling effort.

Future work will focus on refining attention alignment and incorporating broader clinical context, such as patient history and age, to more closely mimic a radiologist's holistic diagnostic process.

Reference: Pathak, S., et al. "Case-level Breast Cancer Prediction for Real Hospital Settings." arXiv:2310.12677v2 [cs.CV], 2024.

Summary updated

Abstract

Breast cancer prediction models for mammography assume that annotations are available for individual images or regions of interest (ROIs), and that there is a fixed number of images per patient. These assumptions do not hold in real hospital settings, where clinicians provide only a final diagnosis for the entire mammography exam (case). Since data in real hospital settings scales with continuous patient intake, while manual annotation efforts do not, we develop a framework for case-level breast cancer prediction that does not require any manual annotation and can be trained with case labels readily available at the hospital. Specifically, we propose a two-level multi-instance learning (MIL) approach at patch and image level for case-level breast cancer prediction and evaluate it on two public and one private dataset. We propose a novel domain-specific MIL pooling observing that breast cancer may or may not occur in both sides, while images of both breasts are taken as a precaution during mammography. We propose a dynamic training procedure for training our MIL framework on a variable number of images per case. We show that our two-level MIL model can be applied in real hospital settings where only case labels, and a variable number of images per case are available, without any loss in performance compared to models trained on image labels. Only trained with weak (case-level) labels, it has the capability to point out in which breast side, mammography view and view region the abnormality lies.
BibTeX
@inproceedings{pathak2023case,
  author = {Shreyasi Pathak and Jörg Schlötterer and Jeroen Geerdink and Jeroen Veltman and Maurice van Keulen and Nicola Strisciuglio and Christin Seifert},
  title = {{Case-level Breast Cancer Prediction for Real Hospital Settings}},
  booktitle = {University of Twente Research Information (Pure Portal)},
  year = {2023},
  doi = {10.48550/arxiv.2310.12677},
}
Figure