TL;DR
This paper introduces a deep learning pipeline to automatically quantify AgNOR scores in histopathology images, a crucial but time-consuming task for cancer prognosis. The developed system achieves performance comparable to human pathologists, significantly reducing the manual effort required.
What Problem Does This Paper Address?
Histopathological analysis often relies on quantifying specific cellular features to predict disease outcomes. Nucleolar Organizer Regions (NORs), visualized as Argyrophilic NORs (AgNORs) after silver staining, are one such feature. The number of AgNORs per nucleus is a known prognostic factor for various tumors, correlating with cell proliferation and tumor growth. However, manually counting AgNORs in hundreds of nuclei per slide is a laborious and time-intensive process for pathologists. This paper aims to automate this process using deep learning, addressing the need for faster, more consistent, and scalable AgNOR scoring in clinical settings.
Methods
The authors frame the AgNOR scoring problem as an object detection task, focusing on the quantity of AgNORs within a nucleus rather than precise segmentation of each individual AgNOR. This approach simplifies the annotation process. They employ an anchor-free, fully convolutional one-stage object detector (FCOS) for this task. To handle the class imbalance in their dataset (where nuclei with few AgNORs are far more common than those with many), they use an ensemble of five detection models trained on different data subsets. For training, they oversample minority classes and use small image patches (128x128 pixels) to better control the class distribution within each patch. The models are trained using the Adam optimizer with a learning rate of 10⁻⁴. For evaluation, the inference process mirrors the manual annotation procedure, dividing images into fields and processing them sequentially.
Datasets
The study utilizes 29 images of canine cutaneous mast cell tumors (CCMCT) stained for AgNORs. These images, each 1569x1177 pixels, were selected by a pathologist from whole slide images and digitized at a resolution of 0.25 microns per pixel. All nuclei within these images were annotated by a pathologist, with the number of identifiable AgNORs serving as the label for each nucleus. This resulted in 23,036 bounding box annotations across twelve classes (including a class for >10 AgNORs). Ten of these images were set aside as a hold-out test set.
Key Insights and Ideas
- Object Detection over Segmentation: Framing AgNOR scoring as an object detection problem (counting) rather than a segmentation problem (outlining each AgNOR) significantly reduces the annotation burden while still capturing the essential prognostic information.
- Ensemble of Detectors for Robustness: Using an ensemble of five FCOS models trained on different data subsets helps to improve the overall robustness and performance of the AgNOR scoring pipeline, especially in the face of data imbalances.
- Class Imbalance Mitigation: The strategies of oversampling minority classes during training and using smaller patch sizes are crucial for effectively training a model on a dataset where the distribution of AgNOR counts is heavily skewed.
- Human-like Performance: The study demonstrates that deep learning models can achieve performance comparable to human experts in this specific histopathological task, opening doors for widespread automation.
Key Results
The developed deep learning pipeline achieved a mean squared error (MSE) of 0.054 and a mean absolute error (MAE) of 0.201 when comparing its AgNOR scores to those of six human pathologists across all images and raters. This indicates a high degree of agreement with expert assessments. At the object level, the model showed satisfactory performance in detecting nuclei with up to 2 AgNORs, achieving mean F1-scores greater than 60%. However, detection accuracy decreased for classes with more than 2 AgNORs per nucleus, with mean F1-scores below 40%, likely due to the scarcity of such instances in the training data.
Limitations and Future Work
The authors acknowledge that the manual annotation process, which involved focusing through individual nuclei at a single z-plane, might not perfectly replicate the full 3D nature of the tissue. They suggest that using image stacks at multiple focus levels and higher scan resolutions could potentially improve the ability of both humans and algorithms to distinguish individual AgNORs. The reduced performance on higher AgNOR count classes is attributed to the significant class imbalance in the training data. Future work could focus on further improving detection for these rarer classes. The authors encourage the use of such automated pipelines for future clinical research to investigate the prognostic significance of AgNORs.
Summary updated