Abstract
We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method has three major components. First, we introduce a novel MIL aggregator that models the relations of the instances in a dual-stream architecture with trainable distance measurement. Second, since WSIs can produce large or unbalanced bags that hinder the training of MIL models, we propose to use self-supervised contrastive learning to extract good representations for MIL and alleviate the issue of prohibitive memory cost for large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI features, and further improve the accuracy of classification and localization. Our model is evaluated on two representative WSI datasets. The classification accuracy of our model compares favorably to fully-supervised methods, with less than 2% accuracy gap across datasets. Our results also outperform all previous MIL-based methods. Additional benchmark results on standard MIL datasets further demonstrate the superior performance of our MIL aggregator on general MIL problems.
Introduction
Whole Slide Images (WSIs) in digital pathology are gigapixel-scale images, often reaching $40,000 \times 40,000$ pixels. Their massive size prevents direct processing by standard neural networks, necessitating their division into thousands of small patches. A significant challenge in WSI classification is the lack of localized annotations; typically, only slide-level labels (e.g., "cancerous" vs. "healthy") are available. This setup is naturally modeled as a Multiple Instance Learning (MIL) problem, where a slide (bag) is positive if it contains at least one cancerous patch (instance). However, identifying these "needles in a haystack" is difficult when positive regions are sparse, often occupying less than 20% of the slide.
Methodology
The proposed DSMIL (Dual-stream Multiple Instance Learning) framework addresses these challenges through a three-stage pipeline: self-supervised feature extraction, a dual-stream aggregator, and multiscale pyramidal fusion.
Self-Supervised Feature Extraction
To avoid the memory constraints of end-to-end training on gigapixel images, the authors employ SimCLR, a self-supervised contrastive learning method. This allows the model to learn rich, domain-specific representations of tissue patches without manual labels. The authors found that features learned via contrastive learning on pathology data significantly outperformed ImageNet-pre-trained features, especially in "unbalanced" bags where the signal is sparse.
Dual-Stream MIL Aggregator
The core innovation is the dual-stream aggregator. It identifies a "critical instance"—the patch most likely to be positive—via a max-pooling stream. A second stream then uses a trainable distance-based attention mechanism to measure the similarity of all other patches to this anchor. This allows the model to "soft-select" relevant patches and ignore the noise of healthy tissue, effectively creating a more refined decision boundary.
Pyramidal Fusion
To capture both cellular detail and tissue architecture, features are extracted at multiple magnifications (e.g., 5x and 20x). These are concatenated into a feature pyramid, mimicking the multi-scale diagnostic process used by human pathologists. While this fusion improved results, the authors noted that adding very low magnifications (like 1.25x) could sometimes reduce discriminative power.
Experimental Results
The model was evaluated on the Camelyon16 (breast cancer metastasis) and TCGA Lung Cancer (sub-type classification) datasets, as well as classical MIL benchmarks like Musk1 and Musk2.
- State-of-the-Art Accuracy: DSMIL achieved 89.9% accuracy on Camelyon16, significantly outperforming previous MIL leaders like ABMIL (84.5%). It narrowed the gap with fully supervised models (which require tedious manual annotations) to less than 2%.
- Superior Localization: Beyond classification, DSMIL was significantly better at highlighting where the cancer was. It achieved a localization FROC score of 0.437, compared to 0.304 for previous RNN-based MIL methods.
- Generalization: The aggregator set new state-of-the-art records on all five classical MIL benchmark datasets (e.g., 93.2% on Musk1), proving its effectiveness beyond medical imaging.
Conclusion
DSMIL bridges the gap between weak and full supervision in digital pathology. By leveraging self-supervised learning and a novel relational aggregator, it achieves near-human diagnostic accuracy and precise localization without the need for expensive pixel-level annotations. While the model handles relations between patches via distance metrics, it does not yet explicitly model spatial or geometric layouts. Future work could integrate graph neural networks or spatial transformers to better capture the structural architecture of the tissue.
Summary updated