Pairwise Relation Learning for Semi-supervised Gland Segmentation

Yutong Xie, Jianpeng Zhang, Zhibin Liao, Chunhua Shen, Johan Verjans, Yong Xia

TL;DR

This paper introduces PRS², a novel semi-supervised learning framework for gland segmentation in histology images. By leveraging pairwise semantic consistency between unlabeled images, PRS² significantly improves segmentation accuracy, especially when labeled data is scarce, and introduces an object-level Dice loss to better handle touching glands.

What Problem Does This Paper Address?

Accurate gland segmentation in histology images is crucial for diagnosing diseases like adenocarcinoma. However, creating detailed pixel-wise annotations for training deep learning models is extremely labor-intensive and costly. Existing deep learning methods, while powerful, demand a large amount of labeled data. This paper tackles the challenge of achieving high-performance gland segmentation with limited labeled data by effectively utilizing abundant unlabeled histology images.

Methods

The proposed framework, PRS², consists of two main modules:

  • Segmentation Network (S-Net): This module is responsible for the actual gland segmentation. It's based on the DeepLabv3+ architecture and is trained on labeled data. A key innovation here is the introduction of a multi-level segmentation loss for S-Net. This loss combines:
  • Pixel-level cross-entropy loss: For general pixel accuracy.
  • Global-level Dice loss: For overall overlap optimization.
  • Object-level Dice loss: A novel contribution, this loss specifically addresses the challenge of separating individual glands that are touching. It measures how well each segmented gland overlaps its corresponding ground truth and vice-versa, promoting instance-level discrimination.
  • Pairwise Relation Network (PR-Net): This unsupervised module works with both labeled and unlabeled data to enhance the image representation learned by the S-Net. PR-Net exploits the semantic consistency between pairs of images in the feature space. It learns to identify similar semantic content across different images, even if they are unlabeled. This learned representation is then transferred to the S-Net through shared encoders, improving its segmentation capabilities.

The PRS² model is trained in two steps: first, S-Net is initialized with labeled data, and then both S-Net and PR-Net are jointly fine-tuned on the combined labeled and unlabeled dataset.

Datasets

The paper evaluates PRS² on two publicly available datasets:

  • GlaS (Gland Segmentation) dataset: This dataset was used in the 2015 MICCAI Gland Segmentation challenge and contains 85 training and 80 test images.
  • CRAG (Colorectal Adenocarcinoma Gland) dataset: This dataset comprises 173 training and 40 test images.

For evaluation, the authors also used the CRAG training set as unlabeled data when evaluating on the GlaS test set, and vice-versa, to simulate realistic semi-supervised scenarios.

Key Insights and Ideas

  • Pairwise Semantic Consistency is Powerful: The core idea that semantic relationships between pairs of images (even unlabeled ones) can be learned and used to improve feature representations is a significant contribution. This goes beyond typical consistency regularization that focuses on perturbations of a single image.
  • Object-level Dice Loss for Instance Separation: The transformation of the object-level Dice metric into a loss function is a clever way to tackle the difficult problem of separating touching glands, which is a common issue in histology images.
  • Semi-Supervised Learning Can Bridge the Data Gap: The results strongly suggest that semi-supervised methods like PRS² can significantly reduce the reliance on large labeled datasets, making advanced segmentation models more accessible.
  • Transfer Learning via Shared Encoders: The parameter sharing between S-Net and PR-Net's encoders is an efficient mechanism for transferring the learned semantic understanding from the unsupervised PR-Net to the supervised S-Net.

Key Results

  • State-of-the-Art Performance: The PRS² model achieved state-of-the-art performance on both the GlaS and CRAG datasets, outperforming several recent supervised and semi-supervised methods.
  • Significant Improvement with Limited Labels: When using only 20% and 50% of the labeled training data, PRS² consistently outperformed its fully-supervised counterpart (S-Net trained with the same limited labels). Notably, PRS² trained with 50% labeled data achieved performance comparable to S-Net trained with 100% labeled data.
  • Effectiveness of Object-Level Dice Loss: Ablation studies showed that the multi-level segmentation loss, including the object-level Dice loss, was crucial for achieving superior performance compared to simpler loss combinations.
  • Efficiency: The model trains in a reasonable time (around 12 hours for training, less than 1 second for inference per image on high-end GPUs).

Limitations and Future Work

  • Computational Cost: While efficient for inference, training involves two networks and a complex loss function, which can be computationally intensive.
  • Generalizability: The paper focuses on gland segmentation. While the pairwise relation learning strategy is presented as generic, its effectiveness on other medical imaging tasks would require further validation.
  • Future Work: The authors suggest that the pairwise relation-based semi-supervised strategy could be applied to other medical image segmentation tasks to reduce the need for extensive annotations. Further exploration of different architectures and loss functions within this framework is also a natural next step.

Summary updated

Abstract

Accurate and automated gland segmentation on histology tissue images is an essential but challenging task in the computer-aided diagnosis of adenocarcinoma. Despite their prevalence, deep learning models always require a myriad number of densely annotated training images, which are difficult to obtain due to extensive labor and associated expert costs related to histology image annotations. In this paper, we propose the pairwise relation-based semi-supervised (PRS^2) model for gland segmentation on histology images. This model consists of a segmentation network (S-Net) and a pairwise relation network (PR-Net). The S-Net is trained on labeled data for segmentation, and PR-Net is trained on both labeled and unlabeled data in an unsupervised way to enhance its image representation ability via exploiting the semantic consistency between each pair of images in the feature space. Since both networks share their encoders, the image representation ability learned by PR-Net can be transferred to S-Net to improve its segmentation performance. We also design the object-level Dice loss to address the issues caused by touching glands and combine it with other two loss functions for S-Net. We evaluated our model against five recent methods on the GlaS dataset and three recent methods on the CRAG dataset. Our results not only demonstrate the effectiveness of the proposed PR-Net and object-level Dice loss, but also indicate that our PRS^2 model achieves the state-of-the-art gland segmentation performance on both benchmarks.
BibTeX
@inproceedings{xie2020pairwise,
  author = {Yutong Xie and Jianpeng Zhang and Zhibin Liao and Chunhua Shen and Johan Verjans and Yong Xia},
  title = {{Pairwise Relation Learning for Semi-supervised Gland Segmentation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59722-1\_40},
}
Figure