TL;DR
This paper introduces WESEG, a novel weakly supervised learning approach for segmenting tumors in whole slide images (WSIs) of various cancers. By leveraging readily available, albeit noisy, slide-level percentage annotations, WESEG achieves superior segmentation performance compared to existing methods, even in challenging out-of-distribution and out-of-domain scenarios.
What Problem Does This Paper Address?
Accurate segmentation of tumors in digital pathology images is crucial for diagnosis and treatment planning. However, creating pixel-level annotations for this task is extremely time-consuming and requires expert pathologists, leading to a scarcity of annotated data. While some approaches use slide-level categorical labels (e.g., "cancer present" or "absent"), these often lack robustness and generalization. This paper tackles the challenge of segmenting tumors across a wide range of cancer types using a weaker form of annotation: the estimated percentage of tumor area within a whole slide image. These percentage estimates are more readily available in clinical practice but are known to be noisy and subject to inter-observer variability.
Methods
The core of WESEG is a weakly supervised multi-instance learning (MIL) framework. It ingeniously uses the slide-level percentage annotations ($p_k$) to construct a "proxy ground-truth" for each training WSI ($x_k$). This proxy map assigns a value of 1 to the $p_k\%$ of image tiles (or pixels) with the highest predicted tumor probability and 0 to the remaining $100-p_k\%$ tiles. This recursive training mechanism ensures that the model learns to align its predicted tumor percentage with the provided annotation.
To account for the inherent noise in percentage annotations, WESEG incorporates parameters ($r_{low}, r_{high}, a_{low}, a_{high}$) that define a margin around the predicted probabilities. This allows for a more robust learning process by discarding probabilities that fall outside a certain range, effectively making the model more resilient to annotation inaccuracies. The framework is designed to be modular and architecture-agnostic, meaning it can be integrated with various deep learning architectures like ResNet, U-Net, or GANs.
Datasets
The study utilizes data from The Cancer Genome Atlas (TCGA), a large public repository containing WSIs from 32 common solid cancer subtypes. For training, a dataset of 18,306 WSIs from 10,903 patients was used, with 3 cancer subtypes held out for testing. Two types of annotations were retrieved from TCGA:
- Binary labels: Indicating whether a WSI is normal (0) or contains neoplastic tissue (1).
- Percentage annotations: Estimating the percentage of tumor cells ("percent_tumor_cells").
For testing, several sets were curated:
- Out-of-distribution testing set: 175 WSIs sampled from the training cohort's hold-out set, with ground-truth tumor maps curated by two pathologists.
- Out-of-location testing sets: 24 WSIs from 3 cancer subtypes (cervix squamous cell, rectum adenocarcinoma, glioblastoma multiforme) not included in the training set.
- Out-of-domain testing sets: WSIs from publicly available datasets (DigestPath for colon adenocarcinoma and PAIP2019 for liver hepatocellular carcinoma) using Formalin-fixed paraffin-embedded (FFPE) slides, which differ from the snap-frozen slides used in TCGA.
Key Insights and Ideas
- Leveraging noisy annotations: The paper demonstrates that even noisy, slide-level percentage annotations can be effectively utilized for accurate tumor segmentation, significantly reducing the need for laborious pixel-level annotations.
- Robustness to annotation bias: The proposed WESEG method shows remarkable performance despite the observed biases in percentage annotations (e.g., tendency to round to multiples of 5% or 20%). This highlights the method's resilience to real-world annotation imperfections.
- Generalization power: WESEG achieves superior performance across various testing scenarios, including out-of-distribution, out-of-location, and out-of-domain datasets, indicating strong generalization capabilities for pan-cancer segmentation.
- Modular and adaptable framework: The architecture-free design of WESEG makes it a versatile tool that can be readily integrated with existing or future segmentation models.
Key Results
WESEG consistently outperformed benchmarked weakly supervised methods, including AttentionMIL and AlphaBetaMIL, as well as a supervised approach that emulated pixel-level annotations.
- On the out-of-distribution testing set, WESEG achieved an AUC of 0.932, outperforming the next best method (Supervised) by 0.027.
- It also showed strong performance on out-of-location (0.946) and out-of-domain (DigestPath: 0.779, PAIP: 0.671) datasets.
- The proposed approach reduced error by over 37% compared to the best method using binary annotations and over 28% compared to the supervised approach using percentages.
- Qualitative results (Figure 1) show WESEG producing segmentation maps that are visually more accurate and complete than those from other methods.
Limitations and Future Work
- Annotation noise: While WESEG is robust to annotation noise, the paper acknowledges that further improvements could be achieved by refining the input percentage annotations, potentially through transductive learning.
- Model-based annotation: The authors suggest that WESEG could be used as an annotation-free refining strategy if the initial percentage annotations were computed by a model trained solely on binary WSI annotations.
- Modality extension: The generic approach could be extended to other imaging modalities beyond whole slide images.
- Bias in annotations: The study identified a tendency for human annotators to round percentage estimates, suggesting that understanding and mitigating such biases is an ongoing area of interest.
Summary updated