Abstract
Automatic methods for early detection of breast cancer on mammography can significantly decrease mortality. Broad uptake of those methods in hospitals is currently hindered because the methods have too many constraints. They assume annotations available for single images or even regions-of-interest (ROIs), and a fixed number of images per patient. Both assumptions do not hold in a general hospital setting. Relaxing those assumptions results in a weakly supervised learning setting, where labels are available per case, but not for individual images or ROIs. Not all images taken for a patient contain malignant regions and the malignant ROIs cover only a tiny part of an image, whereas most image regions represent benign tissue. In this work, we investigate a two-level multi-instance learning (MIL) approach for case-level breast cancer prediction on two public datasets (1.6k and 5k cases) and an in-house dataset of 21k cases. Observing that breast cancer is usually only present in one side, while images of both breasts are taken as a precaution, we propose a domain-specific MIL pooling variant. We show that two-level MIL can be applied in realistic clinical settings where only case labels, and a variable number of images per patient are available. Data in realistic settings scales with continuous patient intake, while manual annotation efforts do not. Hence, research should focus in particular on unsupervised ROI extraction, in order to improve breast cancer prediction for all patients.
Introduction
Bridging the gap between pristine research datasets and the complex reality of clinical data is a critical "last-mile" problem in medical AI. While many models excel on curated datasets with pixel-level annotations, their performance often falters in hospital settings where data is unstructured and labels are sparse. This paper addresses these challenges by proposing a framework that operates under realistic constraints: using only case-level labels and handling a variable number of images per patient.
The Challenge of Realistic Clinical Data
In a standard AI research setting, models are typically trained on "strong" labels where radiologists have manually circled tumors (ROIs) or labeled individual images. However, in a real hospital workflow, the only reliable ground truth is often the final pathology report for the entire case. This environment presents three major challenges:
- Label Sparsity: A positive case-level label does not specify which of the multiple images contains the malignancy.
- Small Targets: Tumors often occupy less than 2% of the total image area, making them easy to overlook in global image analysis.
- Variable Inputs: While a standard exam consists of four views, clinical practice often includes extra "exaggerated" views (e.g., XCCL) to resolve uncertainties. Most traditional models require a fixed input size and cannot accommodate these variations.
Methodology: Two-Level Multi-Instance Learning
The authors propose a two-level Multi-Instance Learning (MIL) framework designed to extract signals from weakly labeled data.
Patch-Level MIL
Each image is decomposed into patches. The model utilizes an unsupervised ROI extraction method to identify potentially malignant regions without explicit spatial guidance during training. This allows the model to focus on the "small targets" inherent in mammography.
Side-wise Pooling
Standard MIL treats all images in a case as an unordered "bag." This work introduces Side-wise Pooling, which groups images by breast side (Left vs. Right). By pooling features for each side independently before final combination, the model mimics the clinical practice of looking for asymmetries between breasts, preventing the dilution of malignant signals by benign tissue from the opposite side.
Dynamic Training
To address variable image counts, a dynamic training procedure was developed. This approach ensures that the model's attention mechanisms remain robust even when specific views are missing or extra views are provided, preventing gradient pollution from unused components.
Experimental Setup and Datasets
The framework was evaluated across three datasets representing varying degrees of scale and realism:
- CBIS-DDSM: A standard public benchmark (1.6k cases).
- VinDr-Mammo: A large public dataset (5k cases).
- CLaM: A massive private dataset from Hospital Group Twente, containing 21,013 cases (over 84,000 images). This dataset reflects the true "real-world" setting, with cases containing between 1 and 7 images.
Results and Discussion
The proposed ES-Attside model outperformed existing case-level baselines. On the private CLaM dataset, it achieved an AUC of 0.85 and an F1-score of 0.61.
Crucially, the study found that case-level labels are sufficient for high-performance prediction. Models trained on these weak labels performed as well as those trained on expensive image-specific labels. Furthermore, qualitative evaluations showed that the model's internal attention mechanisms successfully localized malignant ROIs in the majority of true positive cases, despite never being trained on spatial annotations.
Conclusion
This research demonstrates that two-level MIL can be effectively applied in clinical settings where only case labels and variable image counts are available. By leveraging domain-specific knowledge like side-wise symmetry and unsupervised ROI extraction, AI can scale with continuous patient intake without the bottleneck of manual annotation. Future work will focus on incorporating cross-view transformers to better model the correlations between different angles of the same breast tissue.
Summary updated