TL;DR
This paper compares two attention-based Multiple Instance Learning (MIL) approaches, AMIL and AdMIL, for analyzing Whole Slide Images (WSIs) in cancer diagnosis. They demonstrate that these models can effectively detect tumors and, more importantly, can highlight specific regions of interest (ROIs) that correlate with TP53 mutations, offering potential for improved diagnostic tools.
What Problem Does This Paper Address?
Whole Slide Images (WSIs) are massive digital representations of microscope slides, crucial for cancer diagnosis. However, their sheer size and the fact that diagnostic labels are typically assigned at the slide-level (not for individual microscopic regions) pose significant challenges for AI analysis. This "weakly supervised" nature means models need to learn from slide-level labels while identifying relevant cellular or tissue features within the image. Specifically, this paper tackles two key problems: (1) accurately classifying WSIs for cancer presence, and (2) identifying specific morphological features associated with genetic mutations (like TP53) within these WSIs, which is a less explored area.
Methods
The authors compare two established weakly supervised Multiple Instance Learning (MIL) frameworks: Attention MIL (AMIL) and Additive MIL (AdMIL). Both approaches treat each WSI as a "bag" of smaller image patches ("instances") and use attention mechanisms to learn which patches are most important for the final slide-level classification.
- AMIL: Uses a weighted average of instance embeddings, where weights are learned by a two-layered neural network. The attention mechanism uses a tanh activation function.
- AdMIL: Modifies the aggregation step by passing each patch embedding multiplied by its attention score through a "patch classifier" to produce a "patch score." These scores are then summed to produce the final bag score. The original AdMIL uses a LeakyReLU activation in its attention layer.
- "Our Additive MIL": The researchers propose a modified AdMIL architecture that uses the tanh activation function in its attention layer, similar to AMIL, while retaining AdMIL's additive aggregation strategy. This modification was explored to see if it could improve performance or interpretability.
The core idea is to leverage attention mechanisms not just for classification but also to generate heatmaps that visualize which image regions contribute most to the model's decision, potentially acting as a "virtual stain" for pathologists.
Datasets
The study utilizes datasets from The Cancer Genome Atlas (TCGA):
- TCGA-LUSC (Lung Squamous Cell Carcinoma): Used for tumor detection. A dataset of 694 slides (347 positive, 347 negative) was curated, with patches extracted at 5x magnification. This magnification was chosen as it's commonly used by pathologists for initial tumor assessment and captures tissue-level information.
- TCGA-BRCA (Invasive Breast Carcinoma): Used for TP53 mutation detection. A dataset of 662 slides (331 positive, 331 negative) was curated. For this task, the models were evaluated at three different magnifications: 5x, 10x, and 20x. This was done to understand how different levels of morphological detail influence the model's ability to detect mutation-related patterns.
Key Insights and Ideas
- Tissue-level vs. Cell-level Information: For tumor detection, 5x magnification (tissue-level) was sufficient for good performance, suggesting that models can learn to identify tumors similarly to how pathologists do at this level.
- Morphology and Mutation Correlation: TP53 mutation detection proved more challenging and was significantly more sensitive to higher magnifications (10x and 20x), where finer cellular morphology is resolved. This highlights the importance of detailed cellular features for understanding genetic alterations.
- Interpretability as a Feature: The attention mechanisms in these MIL models are not just for classification; they generate heatmaps that can highlight regions of interest (ROIs). This "slide painting" capability is a key insight, offering a way to visualize why a model makes a certain prediction, which is highly valuable for pathologists.
- Modified AdMIL Performance: The researchers' modified AdMIL architecture (using tanh activation) showed competitive performance, particularly on the TP53 mutation task at 20x magnification, suggesting that architectural tweaks can yield improvements.
- Attention Sparsity: Different attention mechanisms (e.g., LeakyReLU vs. tanh) lead to varying degrees of "sparsity" in attention scores. Sparser attention might be better for scattered tumors, while broader attention might be needed for more diffuse patterns.
Key Results
- Tumor Detection: Both AMIL and AdMIL achieved high performance, with Area Under the Receiver Operating Characteristic Curve (AUROC) scores around 0.95-0.97. The original AMIL achieved the best AUROC of 0.971 ± 0.015, closely followed by their modified AdMIL at 0.963 ± 0.020.
- TP53 Mutation Detection: Performance was significantly lower and highly dependent on magnification.
- At 5x, performance was poor (AUROC around 0.51-0.60), indicating insufficient detail.
- At 10x, performance improved (AUROC around 0.54-0.71), with AMIL leading at 0.711 ± 0.033.
- At 20x, AMIL remained strong (AUROC 0.704 ± 0.040), but their modified AdMIL showed a notable improvement, reaching 0.642 ± 0.029, outperforming the original AdMIL at this level.
- ROI Visualization: Heatmaps were generated for both tasks. For tumor detection, they showed tissue-level patterns. For TP53 mutation detection, higher magnifications revealed more localized patterns, with their modified AdMIL showing a more focused attention at 20x.
Limitations and Future Work
- Dataset Noise: The researchers acknowledge that using flash-frozen slides (which can have artifacts) and random sampling of tiles for the TP53 mutation task might have introduced noise, potentially impacting model performance.
- Expert Evaluation of ROIs: A significant limitation is the lack of qualitative evaluation of the generated heatmaps by expert pathologists. The authors suggest this is crucial for validating the clinical utility of the ROIs.
- Fine-tuning: The models were evaluated in their original configurations without extensive fine-tuning, which could potentially improve performance.
- Broader Architecture Comparison: The study focused on two specific MIL architectures. Future work could involve comparing these with other MIL approaches, including those with self-attention mechanisms.
- Ground Truth for ROIs: Establishing ground truth for ROIs would be a laborious but valuable next step for quantitatively assessing the quality of the attention-based localization.
Summary updated