HistoSegCap: Capsules for Weakly-Supervised Semantic Segmentation of Histological Tissue Type in Whole Slide Images

Mobina Mansoori, Sajjad Shahabodini, Jamshid Abouei, Arash Mohammadi, Konstantinos N. PlataniotisarXiv 2024 · OpenReview

TL;DR

This paper introduces HistoSegCap, a novel method for automatically segmenting tissue types in whole slide images (WSIs) using Capsule Networks and Weakly Supervised Semantic Segmentation (WSSS). By leveraging the strengths of Capsule Networks for spatial relationships and WSSS for reduced annotation burden, HistoSegCap achieves state-of-the-art performance, paving the way for more efficient and accurate digital pathology analysis.

What Problem Does This Paper Address?

Histopathological analysis of Whole Slide Images (WSIs) is crucial for diagnosing diseases, but the sheer size and complexity of these images pose significant challenges for pathologists. Manually identifying diagnostically relevant regions (ROIs) is time-consuming and prone to human error, with reported discrepancies in diagnoses. While Computer-Aided Diagnosis (CAD) systems aim to assist pathologists, a major bottleneck is the lack of pixel-level annotated data, which is expensive and labor-intensive to create. Existing methods often struggle with the fine spatial details and relationships within these large images. This paper tackles the problem of achieving accurate semantic segmentation of histological tissue types in WSIs using a weakly supervised approach, minimizing the need for extensive pixel-level annotations.

Methods

The core innovation of HistoSegCap lies in its use of Capsule Networks for histopathological image analysis, a relatively new neural network architecture that excels at preserving spatial relationships and object orientation. Unlike traditional Convolutional Neural Networks (CNNs) that can lose fine spatial information through pooling operations, Capsule Networks represent features as vectors, where the vector's orientation encodes spatial information and its length encodes the probability of an object's presence.

HistoSegCap employs a Weakly Supervised Semantic Segmentation (WSSS) strategy. This means it learns to segment images using only image-level or coarse annotations, rather than requiring precise pixel-level masks for every tissue type. This significantly reduces the annotation effort.

The proposed architecture consists of several key stages:

  • Patch-Level Classification: Initial classification of image patches to predict potential tissue types.
  • Pixel-Level Reconstruction: A reconstruction layer is used to identify the locations of different tissue labels within the image. This acts as a regularization technique and helps in refining predictions.
  • Pixel-Level Segmentation: This stage generates pixel-level activation maps.
  • Non-Relevant Parts Elimination: This step removes background and other non-functional tissue regions to focus on relevant structures.
  • Fusion Interpretation Approach: The outputs from the reconstruction and a gradient-based interpretation method (SmoothGrad) are fused to produce the final segmentation mask. SmoothGrad is used to mitigate noise and improve the stability of saliency maps.

Datasets

The primary dataset used for training and evaluation is the Atlas of Digital Pathology (ADP) dataset. This dataset comprises 100 H&E-stained WSIs, digitized at a resolution of 0.0625 μm. The dataset is annotated with a hierarchical tissue taxonomy, categorizing tissues based on both morphological and functional aspects. A subset of 43 hand-segmented images from the ADP dataset was used for fine-tuning and evaluation.

Additionally, the model was evaluated on the Warwick-QU dataset, which contains healthy and diseased colon gland images with varying cancer grades. This dataset was used to assess the model's ability to detect diseased tissues.

Key Insights and Ideas

  • Capsule Networks for Histopathology: The paper demonstrates that Capsule Networks are a powerful tool for histopathological image analysis, outperforming traditional CNNs in capturing spatial relationships crucial for accurate tissue segmentation.
  • WSSS with Capsules is Viable: HistoSegCap is the first application of Capsule Networks for Weakly Supervised Semantic Segmentation in this domain, showing that this combination can achieve high performance with reduced annotation burden.
  • Reconstruction as Regularization: The reconstruction layer plays a vital role not only in generating visual representations of tissue locations but also as a regularization technique to improve the overall segmentation accuracy and robustness.
  • Fusion of Interpretation Methods: Combining gradient-based saliency maps (SmoothGrad) with reconstruction-based outputs leads to improved diagnostic quality by reinforcing common areas and providing a more comprehensive understanding of tissue features.
  • Potential for CAD Systems: The success of HistoSegCap highlights its potential to significantly enhance CAD systems, aiding pathologists in faster and more accurate diagnoses.

Key Results

  • HistoSegCap achieved superior performance compared to existing WSSS methodologies on the ADP dataset, outperforming methods like HistoSegNet, SEC, and DSRG in terms of mean Intersection-over-Union (mIoU).
  • On the ADP dataset, HistoSegCap achieved an mIoU of 0.5675 for functional types and 0.2587 for morphological types.
  • In a comparison with HistoSegNet on the ADP dataset, HistoSegCap achieved an accuracy of 95.2% compared to HistoSegNet's 94.3%.
  • Qualitative results showed strong concordance between predicted and ground truth segmentations, capturing fine textural details.
  • On the Warwick-QU dataset, HistoSegCap demonstrated the ability to segment relevant tissues and showed that its segmentation confidence decreases with increasing tumor grade, suggesting its potential as a predictive indicator of disease severity.

Limitations and Future Work

  • Performance Degradation with Tumor Grade: The paper notes that on the Warwick-QU dataset, HistoSegCap's pixel-level predictions become less confident and accurate as the tumor grade worsens. This suggests that the model's performance might be sensitive to the severity of disease.
  • Computational Cost: While Capsule Networks offer advantages, their iterative routing mechanism can be computationally intensive. The paper does not explicitly detail the computational time or efficiency gains/losses compared to CNN-based methods.
  • Interpretation of Dynamic Routing: The paper mentions that interpreting the dynamic routing mechanism of Capsule Networks is challenging, and while they propose an interpretation method, further work in this area could be beneficial.
  • Future Work: The authors suggest that further investigation into the interpretation of dynamic routing and exploring more advanced WSSS techniques in conjunction with Capsule Networks could lead to even greater advancements in histopathological image analysis. They also imply that the model's ability to predict disease severity warrants further exploration.

Summary updated

Abstract

Digital pathology involves converting physical tissue slides into high-resolution Whole Slide Images (WSIs), which pathologists analyze for disease-affected tissues. However, large histology slides with numerous microscopic fields pose challenges for visual search. To aid pathologists, Computer Aided Diagnosis (CAD) systems offer visual assistance in efficiently examining WSIs and identifying diagnostically relevant regions. This paper presents a novel histopathological image analysis method employing Weakly Supervised Semantic Segmentation (WSSS) based on Capsule Networks, the first such application. The proposed model is evaluated using the Atlas of Digital Pathology (ADP) dataset and its performance is compared with other histopathological semantic segmentation methodologies. The findings underscore the potential of Capsule Networks in enhancing the precision and efficiency of histopathological image analysis. Experimental results show that the proposed model outperforms traditional methods in terms of accuracy and the mean Intersection-over-Union (mIoU) metric.
BibTeX
@misc{mansoori2024histosegcap,
  author = {Mobina Mansoori and Sajjad Shahabodini and Jamshid Abouei and Arash Mohammadi and Konstantinos N. Plataniotis},
  title = {{HistoSegCap: Capsules for Weakly-Supervised Semantic Segmentation of Histological Tissue Type in Whole Slide Images}},
  year = {2024},
  eprint = {2402.10851},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.10851},
}
Figure