HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images

Lyndon Chan, Mahdi S. Hosseini, Corwyn Rowsell, Konstantinos N. Plataniotis, Savvas DamaskinosICCV 2019 · OpenReview

Abstract

In digital pathology, tissue slides are scanned into Whole Slide Images (WSI) and pathologists first screen for diagnostically-relevant Regions of Interest (ROIs) before reviewing them. Screening for ROIs is a tedious and time-consuming visual recognition task which can be exhausting. The cognitive workload could be reduced by developing a visual aid to narrow down the visual search area by highlighting (or segmenting) regions of diagnostic relevance, enabling pathologists to spend more time diagnosing relevant ROIs. In this paper, we propose HistoSegNet, a method for semantic segmentation of histological tissue type (HTT). Using the HTT-annotated Atlas of Digital Pathology (ADP) database, we train a Convolutional Neural Network on the patch annotations, infer Gradient-Weighted Class Activation Maps, average overlapping predictions, and post-process the segmentation with a fully-connected Conditional Random Field. Our method out-performs more complicated weakly-supervised semantic segmentation methods and can generalize to other datasets without retraining.

Introduction

The digitization of glass slides into Whole Slide Images (WSIs) has revolutionized pathology, yet it presents a significant "needle in a haystack" challenge. WSIs often contain billions of pixels, and pathologists spend upwards of 57% of their time screening these massive files for diagnostically relevant regions. This manual process is not only tedious but also prone to fatigue-induced errors.

While automated tools exist, they are frequently limited to specific diseases or organs. A truly effective clinical aid must recognize a broad spectrum of histological tissue types (HTTs) across various anatomical sites. However, obtaining the pixel-level ground truth required for traditional supervised learning is prohibitively expensive. HistoSegNet addresses this by utilizing a weakly-supervised framework that learns from easily obtainable patch-level labels.

Methods

HistoSegNet employs a Weakly-Supervised Semantic Segmentation (WSSS) approach, training on patches labeled with the presence of specific tissue types. The pipeline is divided into four primary stages:

  1. Patch-level Classification: A modified VGG16 architecture, optimized for spatial resolution by removing deeper layers and adding batch normalization, identifies the HTTs present in each patch.
  2. Pixel-level Activation: The model generates Gradient-Weighted Class Activation Maps (Grad-CAM) to identify the pixels most contributing to the classification.
  3. Inter-HTT Adjustments: A novel "Class-Specific Grad-CAM" step resolves overlaps between tissue types and synthesizes activation maps for "Background" and "Other" categories.
  4. Post-Processing: A fully-connected Conditional Random Field (CRF) refines the segmentation boundaries, aligning them with the visual textures and edges of the tissue.
The HistoSegNet pipeline for weakly-supervised semantic segmentation of whole slide images.

Results and Discussion

The framework was evaluated using the Atlas of Digital Pathology (ADP) and the GlaS (Gland Segmentation) Challenge datasets. HistoSegNet significantly outperformed existing WSSS baselines, such as SEC and DSRG. For functional tissue types, it achieved a mean Intersection over Union (mIoU) of 0.5505, nearly doubling the performance of the SEC baseline (0.3225).

Comparison of the original tissue patch with its morphological and functional segmentation outputs.

Beyond accuracy, the system is highly efficient. It can process a standard slide containing over 3,300 patches in approximately 1.58 seconds on a consumer-grade GPU, making it viable for real-time clinical use. Qualitative validation by a gastrointestinal pathologist confirmed high concordance between the model's segments and actual biological structures like exocrine glands.

An interesting finding was the relationship between model confidence and tissue health. The authors observed that prediction confidence decreases as tissue becomes more diseased (e.g., high-grade tumors). This suggests that uncertainty in the model's output could serve as a secondary diagnostic signal for identifying pathological abnormalities.

Conclusion

HistoSegNet provides a scalable and efficient solution for segmenting diverse tissue types in WSIs using only weak supervision. While the model currently faces challenges in distinguishing between highly similar tissues and identifying nerve axons, its ability to generalize across datasets and its high processing speed mark a significant step toward practical AI-assisted pathology. Future work will focus on incorporating diseased samples to improve robustness in late-stage cancer diagnostics.

Summary updated

Abstract

In digital pathology, tissue slides are scanned into Whole Slide Images (WSI) and pathologists first screen for diagnostically-relevant Regions of Interest (ROIs) before reviewing them. Screening for ROIs is a tedious and time-consuming visual recognition task which can be exhausting. The cognitive workload could be reduced by developing a visual aid to narrow down the visual search area by highlighting (or segmenting) regions of diagnostic relevance, enabling pathologists to spend more time diagnosing relevant ROIs. In this paper, we propose HistoSegNet, a method for semantic segmentation of histological tissue type (HTT). Using the HTT-annotated Atlas of Digital Pathology (ADP) database, we train a Convolutional Neural Network on the patch annotations, infer Gradient-Weighted Class Activation Maps, average overlapping predictions, and post-process the segmentation with a fully-connected Conditional Random Field. Our method out-performs more complicated weakly-supervised semantic segmentation methods and can generalize to other datasets without retraining.
BibTeX
@inproceedings{chan2019histosegnet,
  author = {Lyndon Chan and Mahdi S. Hosseini and Corwyn Rowsell and Konstantinos N. Plataniotis and Savvas Damaskinos},
  title = {{HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=9x_bXg_Y3qYP},
}
Figure