Deep Learning-Based Automatic Assessment of AgNOR-scores in Histopathology Images

Jonathan Ganz, Karoline Lipnik, Jonas Ammeling, Barbara Richter, Chloé Puget, Eda Parlak, Laura Diehl, Robert Klopfleisch, Taryn A. Donovan, Matti Kiupel, Christof A. Bertram, Katharina Breininger, Marc Aubreville

TL;DR

Counting argyrophilic nucleolar organizer regions (AgNORs) inside cell nuclei is a powerful prognostic tool for tumor malignancy, but doing it manually is incredibly tedious and subjective. This paper introduces a deep learning pipeline that reframes AgNOR scoring from a complex segmentation task into a practical object detection problem. By classifying entire nuclei based on their internal AgNOR counts, the model achieves human-equivalent performance, matching a consensus panel of six expert pathologists with a mean squared error of just 0.054.

What Problem Does This Paper Address?

Nucleolar organizer regions (NORs) are segments of DNA actively involved in RNA transcription. When stained with silver-based methods, they appear as dark dots called AgNORs. The average number of AgNORs per nucleus serves as a critical biomarker: more numerous, smaller AgNORs typically signify higher cell proliferation and tumor malignancy.

In clinical practice, calculating this "AgNOR-score" is a grueling task. Pathologists must manually count AgNORs in at least 100 randomly selected nuclei per tissue section, constantly adjusting the microscope's focus. Beyond being labor-intensive, it suffers from high inter-rater variability. While automation has been attempted for cytological samples (where cells are cleanly separated on a bright background), doing this on complex histopathological tissue sections—where cell borders are obscured by dense, dark stroma—remains an open challenge.

Methods

Instead of trying to segment the exact boundaries of every tiny AgNOR dot (which would require an impossible amount of pixel-level annotation effort), the authors make a clever pivot: they treat the problem as an object detection task.

Fig. 1. The left side of Figure a) shows a CCMCT section stained with silver-based staining like described in [3]. The grid used in the experiment is overlaid. On the right side of Figure a), a detailed representation of two nuclei with marked AgNORs is displayed. In Figure b), fields from randomly selected cases of the hold-out test set are shown.

As shown in Figure 1, the target object to detect is the cell nucleus, and its "class label" is simply the integer count of AgNORs inside it (ranging from 0 to 10, and a final class for $>10$).

The pipeline consists of:

  • Core Architecture: FCOS (Fully Convolutional One-Stage object detector), a state-of-the-art, anchor-free object detection model.
  • Ensemble Strategy: To combat severe class imbalance, they train an ensemble of five FCOS models on different random subsets of the data and use the median prediction to determine the final score.
  • Training Optimizations: Models are trained on small $128 \times 128$ pixel patches to maintain tight control over class distributions, combined with oversampling of rare, high-count AgNOR classes.
  • Human-in-the-Loop Inference: To ensure a fair comparison with human experts, the algorithm processes the tissue image field-by-field using a 12-field grid until it hits a minimum threshold of 100 detected nuclei.

Datasets

The authors utilized a dataset of 29 high-resolution images ($1569 \times 1177$ pixels) of canine cutaneous mast cell tumors (CCMCT), stained using a silver-based histochemical method.

  • Annotations: A pathologist annotated every single nucleus, yielding 23,036 bounding boxes split across 12 classes.
  • Data Split: 19 images were used for training, and 10 images were held out as a test set.
  • Characteristics: The dataset exhibits a massive class imbalance, heavily skewed toward cells with fewer than four AgNORs per nucleus.

Key Insights and Ideas

  • Quantity > Shape: The fundamental insight is that clinical prognosis relies entirely on the quantity of AgNORs, not their precise morphology. Reframing the task from pixel-level segmentation to count-based object detection drastically simplifies the data annotation pipeline without losing clinical utility.
  • The Illusion of Human Ground Truth: The human rater experiment (involving 6 pathologists) revealed only "moderate" agreement (Fleiss' Kappa of 0.545). As seen in Figure 2a, variance among humans spikes dramatically in highly malignant cases (e.g., Case 3), where AgNORs are smaller, more numerous, and physically harder to distinguish.
  • Imbalance Resiliency: While the model's performance drops significantly when classifying rare, high-count nuclei (as shown by the lower F1-scores for classes $\ge 3$ in Figure 2b), this has a negligible impact on the final image-level AgNOR-score because these high-count cells are statistically rare in the overall cell population.
AgNOR scores per image and F1-score of detectors on class level

Key Results

  • Human-Like Accuracy: The mean squared error (MSE) between the algorithmically determined AgNOR-scores and the human expert consensus was 0.054, with a mean absolute error (MAE) of 0.201.
  • Class-Level Performance: The ensemble achieved strong performance on the most common classes (0, 1, and 2 AgNORs per nucleus), yielding mean F1-scores greater than 60%.
  • Robustness: The model successfully tracked the overall upward and downward trends across all 10 distinct test cases, closely mirroring the average human consensus line (Figure 2a).

Limitations and Future Work

  • The Single Z-Plane Problem: The digital slides used in this study were captured on a single focal plane. In a real clinical setting, pathologists manually scroll up and down through the tissue depth (the Z-axis) to spot hidden AgNORs. This single-plane limitation likely caused both the humans and the model to undercount AgNORs.
  • Future Direction (Z-Stacks): The authors suggest that future iterations should utilize Z-stacks (3D volumes composed of images stitched together at multiple focal lengths) to enable digital post-hoc focusing, which would likely boost accuracy for both algorithms and human experts.
  • Mitigating Class Imbalance: The model still exhibits a slight tendency to underpredict higher AgNOR scores due to data scarcity in those classes, a limitation that could be addressed with targeted data collection or advanced generative data augmentation.

Summary updated

Abstract

Nucleolar organizer regions (NORs) are parts of the DNA that are involved in RNA transcription. Due to the silver affinity of associated proteins, argyrophilic NORs (AgNORs) can be visualized using silver-based staining. The average number of AgNORs per nucleus has been shown to be a prognostic factor for predicting the outcome of many tumors. Since manual detection of AgNORs is laborious, automation is of high interest. We present a deep learning-based pipeline for automatically determining the AgNOR-score from histopathological sections. An additional annotation experiment was conducted with six pathologists to provide an independent performance evaluation of our approach. Across all raters and images, we found a mean squared error of 0.054 between the AgNOR- scores of the experts and those of the model, indicating that our approach offers performance comparable to humans.
BibTeX
@inproceedings{ganz2023deep,
  author = {Jonathan Ganz and Karoline Lipnik and Jonas Ammeling and Barbara Richter and Chloé Puget and Eda Parlak and Laura Diehl and Robert Klopfleisch and Taryn A. Donovan and Matti Kiupel and Christof A. Bertram and Katharina Breininger and Marc Aubreville},
  title = {{Deep Learning-Based Automatic Assessment of AgNOR-scores in Histopathology Images}},
  booktitle = {Bildverarbeitung für die Medizin},
  year = {2023},
  doi = {10.1007/978-3-658-41657-7\_49},
}
Figure