Highly Efficient Follicular Segmentation in Thyroid Cytopathological Whole Slide Image

Siyan Tao, Yao Guo, Chuang Zhu, Huang Chen, Yue Zhang, Jie Yang, Jun Liu

TL;DR

Researchers have developed a hybrid deep learning model that speeds up the analysis of massive thyroid cytopathological Whole Slide Images (WSIs) by 93%. By integrating a classification branch to skip irrelevant areas and enhancing the DeepLab V3 architecture with fine-grained features, the system achieves high segmentation accuracy while making automated thyroid cancer diagnosis computationally feasible.

What Problem Does This Paper Address?

Thyroid cancer diagnosis often relies on Fine Needle Aspiration (FNA), where clinicians examine tissue slides under a microscope. To automate this, "Whole Slide Images" (WSIs) are used. However, these digital slides are gargantuan—often exceeding 210,000 x 140,000 pixels.

Processing these images presents two major hurdles:

  1. Computational Cost: The sheer size makes it impossible to load a full slide into memory, and processing every single pixel is incredibly slow.
  2. Data Imbalance: In a typical thyroid slide, less than 10% of the area actually contains the follicular cells necessary for diagnosis. The rest is "junk" data—either blank space or "colloid" (a gelatinous substance) that provides no diagnostic value.

Methods

The authors propose a multi-pronged approach to make segmentation both accurate and efficient:

  • Hybrid Architecture: Instead of running a heavy segmentation model over the entire slide, they built a hybrid model based on ResNet-101. It features a classification branch that shares the first block of layers with the segmentation model. This branch acts as a gatekeeper, quickly identifying if a patch is "Follicular," "Colloid," or "Non-info." If it's not follicular, the model skips the expensive segmentation step entirely.
  • E-ASPP (Enhanced Atrous Spatial Pyramid Pooling): Standard DeepLab V3 uses ASPP to capture multi-scale information but often loses fine details. The authors introduced "Enhanced ASPP," which merges low-scale, high-resolution features from earlier layers (Block 3) back into the final pooling stage to better represent tiny cellular details.
  • Criterion-Oriented Adaptive Loss Function: To speed up training, they introduced a novel loss function that weights the standard cross-entropy loss by a performance criterion (like Mean Intersection over Union). This forces the model to converge faster by focusing on the metrics that actually matter for evaluation.

Datasets

The study used a private clinical dataset provided by a top-level comprehensive hospital:

  • 15 Whole Slide Images (WSIs): These were collected from real patients and stained using the Pap method.
  • Split: 13 WSIs were used to generate a "patch dataset" for training and preliminary testing, while the remaining 2 WSIs were reserved for final "WSI-level" performance testing.
  • Preprocessing: They applied a color adjustment method to normalize the significant staining variations often found between different pathology slides.

Key Insights and Ideas

  • The "Skip" Strategy: The most significant efficiency gain comes from the realization that you don't need to segment what you've already classified as irrelevant. By filtering out >90% of the image via the classification branch, the "heavy lifting" is reserved for the areas that matter.
  • Feature Re-use: Sharing the initial layers (Block 1 of ResNet-101) between the classifier and the segmenter minimizes the parameter count and ensures the model learns a robust base representation for both tasks.
  • Adaptive Loss as a Catalyst: The authors found that dividing the loss by the current performance metric (e.g., mIoU) acts as a dynamic learning rate of sorts, pushing the model to improve more aggressively when performance is low.

Key Results

  • Efficiency: The proposed method reduced the time required for WSI segmentation by 93%. While standard models like FCN or DeepLab V3 took nearly 90 minutes (5300–5800 seconds) to process a slide, this hybrid model finished in roughly 12.6 minutes (756 seconds).
  • Segmentation Accuracy: On individual patches, the model achieved 80.9% mIoU (Mean Intersection over Union), outperforming FCN (80.2%) and standard DeepLab V3 (72.4%).
  • Classification Accuracy: The integrated classifier reached 98% accuracy, ensuring that very few follicular areas were accidentally skipped.
  • WSI-Level Performance: Even on complex, full-sized slides, the model maintained a 53.4% mIoU, which the authors note is superior to existing models when applied to real-world clinical "Whole Slide" scenarios.

Limitations and Future Work

  • Dataset Size: The study was conducted on a relatively small sample of 15 WSIs. Testing on a larger, multi-institutional dataset would be necessary to prove the model's generalizability.
  • WSI-Level Drop: There is a noticeable drop in mIoU from the patch level (80.9%) to the WSI level (53.4%). This suggests that while the model is fast, the transition from controlled patches to "in-the-wild" whole slides still introduces complexities (like edge artifacts or varied cell densities) that need further refinement.
  • Hardware Requirements: While efficient, the training still required high-end GPUs (2x NVIDIA GTX 1080Ti), which might not be available in all clinical settings.

Summary updated

Abstract

In this paper, we propose a novel method for highly efficient follicular segmentation of thyroid cytopathological WSIs. Firstly, we propose a hybrid segmentation architecture, which integrates a classifier into Deeplab V3 by adding a branch. A large amount of the WSI segmentation time is saved by skipping the irrelevant areas using the classification branch. Secondly, we merge the low scale fine features into the original atrous spatial pyramid pooling (ASPP) in Deeplab V3 to accurately represent the details in cytopathological images. Thirdly, our hybrid model is trained by a criterion-oriented adaptive loss function, which leads the model converging much faster. Experimental results on a collection of thyroid patches demonstrate that the proposed model reaches 80.9 $$\%$$ on the segmentation accuracy. Besides, 93 $$\%$$ time is reduced for the WSI segmentation by using our proposed method, and the WSI-level accuracy achieves 53.4 $$\%$$ .
BibTeX
@inproceedings{tao2020highly,
  author = {Siyan Tao and Yao Guo and Chuang Zhu and Huang Chen and Yue Zhang and Jie Yang and Jun Liu},
  title = {{Highly Efficient Follicular Segmentation in Thyroid Cytopathological Whole Slide Image}},
  booktitle = {Precision Health and Medicine},
  year = {2020},
  url = {https://openreview.net/forum?id=rosZx9v-h-x},
}

Loading related content...

Figure