Qualitative Histopathological Classification of Primary Bone Tumors Using Deep Learning: A Pilot Study

Yuzhang Tao, Xiao Huang, Yiwen Tan, Hongwei Wang, Weiqian Jiang, Yu Chen, Chenglong Wang, Jing Luo, Zhi Liu, Kangrong Gao, Yang Wu, Minkang Guo, Boyu Tang, Aiguo Zhou, Mengli Yao, Tingmei Chen, Youde Cao, Chengsi Luo, Jian ZhangFrontiers in Oncology 2021 · DOI

TL;DR

This study introduces a deep learning (DL) approach for classifying primary bone tumors based on their aggressiveness from histopathological images. The DL models, particularly VGG-16 and Inception V3, achieved performance comparable to senior pathologists, offering a promising tool to aid in diagnosis and expedite clinical workflows.

What Problem Does This Paper Address?

Diagnosing bone tumors from histopathology is a complex task for pathologists. This is due to the relatively low incidence of these tumors, leading to less experience among general pathologists, and the inherent challenge of distinguishing between different bone tumor types that can share similar microscopic appearances. The subjective nature of human interpretation further complicates accurate and consistent classification, which is crucial for determining the tumor's aggressiveness (benign, intermediate, or malignant) and guiding appropriate treatment. While AI has shown success in classifying epithelial tumors, its application to the more diverse and mesenchymal origins of bone tumors remains less explored.

Methods

The researchers employed a deep learning pipeline using convolutional neural networks (CNNs) for histopathological classification. They started with 427 whole slide images (WSIs) of bone tumors, which were meticulously annotated by pathologists to identify regions of interest (ROIs) containing tumor cells. These ROIs were then cropped into numerous 256x256 pixel image patches.

Six widely-used CNN architectures (AlexNet, VGG-16, Inception V3, DenseNet-121, ResNet-50, and MnasNet) were trained on these patches for both binary (benign vs. non-benign) and ternary (benign vs. intermediate vs. malignant) classification. Data augmentation techniques were applied to enhance model robustness.

To evaluate performance, the models were tested on a separate dataset. For slide-level prediction, the average prediction of all patches from a single WSI was used. The performance of the best-performing DL models (VGG-16 and Inception V3) was then directly compared against the diagnoses provided by four pathologists with varying levels of experience. Gradient-weighted class activation mapping (Grad-CAM) was utilized to visualize which image regions the DL models focused on for their predictions, offering insights into their decision-making process.

Datasets

The study utilized a dataset of 427 pathological slides of primary bone tumors, classified into benign, intermediate, and malignant categories. These slides were scanned to create whole slide images (WSIs). From these WSIs, a total of 716,838 image patches of 256x256 pixels were extracted for training and validation. The dataset was split into training, validation, and testing sets in a 70:15:15 ratio. This dataset, derived from the First Affiliated Hospital of Chongqing Medical University, represents a newly curated benchmark for DL-based bone tumor classification.

Key Insights and Ideas

  • DL's Potential for Bone Tumors: The study demonstrates that deep learning models can effectively classify bone tumors based on aggressiveness, a task that is challenging even for experienced pathologists.
  • VGG-16 and Inception V3 Shine: Among the tested architectures, VGG-16 and Inception V3 consistently outperformed others in both patch-level and slide-level classification tasks.
  • Pathologist-Level Performance: The best DL models achieved performance metrics comparable to, and in some cases exceeding, those of senior pathologists, suggesting their potential as valuable diagnostic aids.
  • Visualizing DL Decisions: Grad-CAM proved instrumental in understanding why the DL models made certain predictions, highlighting specific histopathological features that contribute to classification and revealing how the models interpret complex visual data.
  • Addressing Subjectivity: DL offers a quantitative and objective approach to classification, potentially mitigating the subjectivity inherent in human interpretation.

Key Results

  • Patch-Level Performance: VGG-16 achieved the highest accuracy (85.96%) and F1-score (87.91%) for binary classification, while Inception V3 showed the best specificity (91.34%) for patch-level tasks. For ternary classification, VGG-16 and Inception V3 also led, with VGG-16 achieving a Cohen's Kappa Score (CKS) of 0.601.
  • Slide-Level Performance: In slide-level binary classification, VGG-16 and Inception V3 achieved excellent Area Under the Curve (AUC) scores of 0.962 and 0.971, respectively. For ternary classification, Inception V3 showed the highest CKS of 0.802, closely followed by VGG-16 with a CKS of 0.731.
  • Comparison with Pathologists: The senior pathologist achieved a CKS of 0.685 for ternary classification, comparable to the DL models. However, attending and junior pathologists showed significantly lower performance (CKS < 0.7). In binary classification, the senior pathologist had the best accuracy (84.62%) among pathologists, but the DL models generally outperformed them in overall metrics.

Limitations and Future Work

The authors acknowledge several limitations:

  • Single-Center Study: The dataset was collected from a single institution, which might limit the generalizability of the models to data from other centers with potentially different staining protocols or WSI scanning techniques.
  • Label Noise: Manual annotation of ROIs can introduce noise (incorrectly labeled patches), which can bias DL training. While averaging predictions helps, it's not a perfect solution.
  • Imbalanced Dataset: The dataset had an uneven distribution of cases across different classifications, potentially leading to selection bias.
  • Rare Cases: The models may not be well-trained for rare bone tumor types due to limited representative patches.
  • Focus on Neoplastic Lesions: The models were trained only on neoplastic lesions and cannot differentiate from non-neoplastic mimics like osteomyelitis.
  • Histopathology Only: The study focused solely on histopathological data, whereas clinical and radiological information are typically used by pathologists for diagnosis.

Future work should involve:

  • Multi-center Studies: Expanding the dataset to include data from multiple institutions to improve model generalizability.
  • Advanced Annotation Techniques: Exploring weakly supervised or unsupervised DL models to reduce reliance on manual annotation and mitigate label noise.
  • Integrating Multi-modal Data: Combining clinical, radiological, and histopathological data to create more comprehensive and robust DL models.
  • Addressing Rare Cases: Including more data on rare bone tumor subtypes to enhance model performance for these specific diagnoses.
  • Stain Normalization and Defect Detection: Developing algorithms to handle variations in stain quality and detect defective slides for a more robust pipeline.

Summary updated

Abstract

DL can effectively classify bone tumors histopathologically in terms of aggressiveness with performance similar to senior pathologists. Our results are promising and would help expedite the future application of DL-assisted histopathological diagnosis for bone tumors.
BibTeX
@inproceedings{tao2021qualitative,
  author = {Yuzhang Tao and Xiao Huang and Yiwen Tan and Hongwei Wang and Weiqian Jiang and Yu Chen and Chenglong Wang and Jing Luo and Zhi Liu and Kangrong Gao and Yang Wu and Minkang Guo and Boyu Tang and Aiguo Zhou and Mengli Yao and Tingmei Chen and Youde Cao and Chengsi Luo and Jian Zhang},
  title = {{Qualitative Histopathological Classification of Primary Bone Tumors Using Deep Learning: A Pilot Study}},
  booktitle = {Frontiers in Oncology},
  year = {2021},
  doi = {10.3389/fonc.2021.735739},
}
Figure