A Comprehensive Review of Computer-aided Whole-slide Image Analysis: from Datasets to Feature Extraction, Segmentation, Classification, and Detection Approaches

Chen Li, Xintong Li, Md Mamunur Rahaman, Xiaoyan Li, Hongzan Sun, Hong Zhang, Yong Zhang, Xiaoqi Li, Jian Wu, Yudong Yao, Marcin GrzegorzekarXiv 2021 · OpenReview

Abstract

With the development of computer-aided diagnosis (CAD) and image scanning technology, Whole-slide Image (WSI) scanners are widely used in the field of pathological diagnosis. Therefore, WSI analysis has become the key to modern digital pathology. Since 2004, WSI has been used more and more in CAD. Since machine vision methods are usually based on semi-automatic or fully automatic computers, they are highly efficient and labor-saving. The combination of WSI and CAD technologies for segmentation, classification, and detection helps histopathologists obtain more stable and quantitative analysis results, save labor costs and improve diagnosis objectivity. This paper reviews the methods of WSI analysis based on machine learning, discussing publicly available datasets, evaluation metrics, and the latest developments in segmentation, classification, and detection.

Introduction

Digital pathology has undergone a radical transformation since the introduction of Whole-slide Imaging (WSI) in the early 2000s. This review provides a systematic roadmap of the field, tracing the evolution of WSI analysis from its infancy in 2004 through the deep learning explosion of 2020. By categorizing over 210 research works, the authors demonstrate the transition from manual feature engineering to sophisticated, automated diagnostic pipelines that handle the unique "gigapixel" challenges of medical imaging.

The Gigapixel Challenge

Traditional pathology is a manual, qualitative process where a pathologist examines glass slides under a microscope. This method is inherently subjective, prone to fatigue, and difficult to reproduce. While digitizing these slides via WSI solves storage and sharing issues, it creates a massive data problem. A single WSI can reach dimensions of 100,000 x 100,000 pixels—far too large for standard computer vision models to process in a single pass. The core of modern CAD research focuses on bridging the gap between this raw digital data and actionable clinical utility through efficient image segmentation, classification, and object detection.

The WSI Analysis Pipeline

The paper identifies a standardized seven-step workflow for processing digital slides. This pipeline begins with medical demands and data acquisition, moving through complex computational stages to reach a final system evaluation.

The comprehensive workflow of computer-aided WSI analysis, from data acquisition to system evaluation.
  1. Data Acquisition: Digitizing slides via area or line scanners.
  2. Image Presentation: Managing the "pyramid" structure of WSIs to handle multiple resolution levels.
  3. Preprocessing: Essential techniques like color normalization to handle staining variations and data augmentation to improve model robustness.
  4. Feature Extraction: A shift from traditional hand-crafted descriptors (color histograms, texture patterns) to automatic feature learning using Convolutional Neural Networks (CNNs) like ResNet and Inception.
  5. Post-processing: Refining raw model outputs through morphological filtering or region removal.
  6. Classifier Design: The evolution from classical SVMs and Random Forests to Deep Learning and Multiple Instance Learning (MIL).
  7. System Evaluation: Utilizing specialized metrics such as Dice coefficients for segmentation and FROC curves for detection.

Benchmarks and Datasets

Progress in the field has been largely driven by the availability of "gold standard" public benchmarks. Key repositories include:

  • TCGA (The Cancer Genome Atlas): A massive repository containing over 470 TB of multi-cancer genomic and WSI data.
  • Camelyon (16/17): The primary benchmark for detecting breast cancer metastasis in lymph nodes.
  • TUPAC16: A dataset focused on tumor proliferation and mitosis detection.
  • Kimia Path24: A collection of 24 different texture patterns used for image retrieval and classification.

Technological Evolution and Performance

Research activity in WSI analysis has seen an exponential increase, particularly after 2016, coinciding with the maturity of deep learning frameworks.

The exponential growth of WSI-related research publications from 2004 to 2020, categorized by task.

Key insights from this evolution include:

  • Multi-Resolution Mimicry: Successful CAD systems mimic human pathologists by using a "top-down" approach—scanning low-resolution versions to find Regions of Interest (ROI) before zooming into high-resolution patches.
  • The Rise of MIL: Because pixel-level annotation of gigapixel images is often impossible, Multiple Instance Learning (MIL) allows models to learn from slide-level labels (e.g., "cancerous") without needing every individual cell labeled.
  • Quantitative Success: Automated systems have improved diagnostic efficiency by roughly 6.6x in certain workflows. Deep learning models like U-Net have achieved Jaccard indices as high as 0.937 for tissue segmentation, while ResNet-based models consistently achieve AUCs above 0.95 for cancer grading.

Future Directions and Limitations

Despite these advances, several hurdles remain. The "Black Box" nature of deep learning lacks the interpretability required for clinical trust. Furthermore, there is a persistent need for larger, fully annotated datasets to move beyond "weak labels." Hardware bottlenecks also remain a concern; future research must focus on lightweight networks that can operate on standard hospital hardware. Finally, models still struggle with "domain shift," where performance drops when applied to images from different scanners or laboratories, making stain normalization and generalization a top priority for future development.

Summary updated

Abstract

With the development of computer-aided diagnosis (CAD) and image scanning technology, Whole-slide Image (WSI) scanners are widely used in the field of pathological diagnosis. Therefore, WSI analysis has become the key to modern digital pathology. Since 2004, WSI has been used more and more in CAD. Since machine vision methods are usually based on semi-automatic or fully automatic computers, they are highly efficient and labor-saving. The combination of WSI and CAD technologies for segmentation, classification, and detection helps histopathologists obtain more stable and quantitative analysis results, save labor costs and improve diagnosis objectivity. This paper reviews the methods of WSI analysis based on machine learning. Firstly, the development status of WSI and CAD methods are introduced. Secondly, we discuss publicly available WSI datasets and evaluation metrics for segmentation, classification, and detection tasks. Then, the latest development of machine learning in WSI segmentation, classification, and detection are reviewed continuously. Finally, the existing methods are studied, the applicabilities of the analysis methods are analyzed, and the application prospects of the analysis methods in this field are forecasted.
BibTeX
@misc{li2021comprehensive,
  author = {Chen Li and Xintong Li and Md Mamunur Rahaman and Xiaoyan Li and Hongzan Sun and Hong Zhang and Yong Zhang and Xiaoqi Li and Jian Wu and Yudong Yao and Marcin Grzegorzek},
  title = {{A Comprehensive Review of Computer-aided Whole-slide Image Analysis: from Datasets to Feature Extraction, Segmentation, Classification, and Detection Approaches}},
  year = {2021},
  eprint = {2102.10553},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.10553},
}
Figure