Nuclear Morphometry using a Deep Learning-based Algorithm has Prognostic Relevance for Canine Cutaneous Mast Cell Tumors

Andreas Haghofer, Eda Parlak, Alexander Bartel, Taryn A. Donovan, Charles-Antoine Assenmacher, Pompei Bolfa, Michael J. Dark, Andrea Fuchs-Baumgartinger, Andrea Klang, Kathrin Jäger, Robert Klopfleisch, Sophie Merz, Barbara Richter, F. Yvonne Schulman, Jonathan Ganz, Josef Scharinger, Marc Aubreville, Stephan M. Winkler, Matti Kiupel, Christof A. Bertram

Abstract

Variation in nuclear size and shape is an important criterion of malignancy for many tumor types; however, categorical estimates by pathologists have poor reproducibility. Measurements of nuclear characteristics (morphometry) can improve reproducibility, but manual methods are time consuming. The aim of this study was to explore the limitations of estimates and develop alternative morphometric solutions for canine cutaneous mast cell tumors (ccMCT). We assessed the following nuclear evaluation methods for measurement accuracy, reproducibility, and prognostic utility: 1) anisokaryosis (karyomegaly) estimates by 11 pathologists; 2) gold standard manual morphometry of at least 100 nuclei; 3) practicable manual morphometry with stratified sampling of 12 nuclei by 9 pathologists; and 4) automated morphometry using a deep learning-based segmentation algorithm. The study dataset comprised 96 ccMCT with available outcome information. Inter-rater reproducibility of karyomegaly estimates was low ($\kappa$ = 0.226), while it was good (ICC = 0.654) for practicable morphometry of the standard deviation (SD) of nuclear size. As compared to gold standard manual morphometry (AUC = 0.839, 95% CI: 0.701 - 0.977), the prognostic value (tumor-specific survival) of SDs of nuclear area for practicable manual morphometry (12 nuclei) and automated morphometry were high with an area under the ROC curve (AUC) of 0.868 (95% CI: 0.737 - 0.991) and 0.943 (95% CI: 0.889 - 0.996), respectively. This study supports the use of manual morphometry with stratified sampling of 12 nuclei and algorithmic morphometry to overcome the poor reproducibility of estimates.

Introduction

In veterinary oncology, the appearance of a tumor's cell nuclei—specifically their variation in size and shape, known as nuclear pleomorphism—is a critical indicator of cancer aggressiveness. For canine cutaneous mast cell tumors (ccMCT), pathologists typically estimate this variation using categorical scales such as "mild," "moderate," or "severe." However, these subjective estimates suffer from significant inter-observer variability. While manual morphometry (measuring 100+ nuclei) provides an objective "gold standard," the time required—often 20 minutes per slide—is prohibitive for routine diagnostic use. This study investigates whether deep learning-based algorithmic morphometry can provide high-accuracy prognostic data with the efficiency required for clinical practice.

Methodology

The researchers compared four distinct approaches to evaluating nuclear pleomorphism in ccMCT:

  1. Categorical Estimates: 11 pathologists performed subjective assessments using two-tier and three-tier systems.
  2. Gold Standard Manual Morphometry: A pathologist manually outlined at least 100 nuclei per case to establish a baseline for objective measurement.
  3. Practicable Manual Morphometry: Pathologists measured 12 representative nuclei using a "stratified sampling" strategy, intentionally selecting small, medium, and large nuclei to capture the range of variation.
  4. Automated Algorithmic Morphometry: A deep learning model utilizing a U-Net++ architecture was developed to automatically segment neoplastic nuclei. This model calculated parameters including mean area, standard deviation (SD) of the area, and shape indicators like eccentricity and solidity.

Data and Model Training

The study relied on two primary datasets:

  • Ground Truth Dataset: Comprising 85 images from four veterinary laboratories, this dataset contained 40,542 manually annotated nuclear outlines. This extensive labeling was used to train and validate the segmentation algorithm.
  • Outcome Dataset: This included 96 ccMCT cases with a minimum of 12 months of clinical follow-up, allowing researchers to correlate nuclear measurements with actual patient survival.

The segmentation model demonstrated high technical performance, achieving a Dice score of 0.785 and an F1 score of 0.854, indicating its ability to accurately identify and outline nuclei even in complex slides containing inflammation or heavy granulation.

Key Findings

The Limitations of Human Estimation

The study confirmed that human "eyeballing" is highly subjective. Inter-rater reproducibility for categorical estimates was "slight to fair" ($\kappa$ = 0.226). In contrast, even the simplified "practicable" manual morphometry of just 12 nuclei showed good reproducibility (ICC = 0.654), suggesting that structured sampling is superior to subjective estimation.

Superior Prognostic Accuracy of AI

The automated algorithmic approach outperformed all other methods in predicting tumor-specific survival. The AI’s measurement of nuclear area variation (SD) achieved an AUC of 0.943. This surpassed the human gold standard (AUC = 0.839) and the traditional clinical benchmark of Mitotic Count (AUC = 0.885). The AI's advantage lies in its ability to analyze thousands of nuclei across the entire slide, capturing tumor heterogeneity that human observers might overlook.

Size vs. Shape

While the study evaluated both nuclear size and shape, parameters related to size variation (specifically the standard deviation of the nuclear area) were found to be much stronger predictors of mortality than shape-based metrics like solidity or eccentricity.

Discussion and Conclusion

Despite the promising results, the authors noted limitations, including a small outcome sample size (13 tumor-specific deaths) and the potential for "domain shift" when applying the model to different slide scanners. Future work will focus on testing the algorithm's robustness across diverse hardware and laboratory preparations.

Ultimately, this research demonstrates that deep learning-based morphometry is a highly reproducible and accurate tool for prognosticating canine mast cell tumors. By transitioning from subjective "art" to data-driven science, algorithmic morphometry offers a path toward more consistent and reliable veterinary cancer diagnostics.

Summary updated

Abstract

Variation in nuclear size and shape is an important criterion of malignancy for many tumor types; however, categorical estimates by pathologists have poor reproducibility. Measurements of nuclear characteristics (morphometry) can improve reproducibility, but manual methods are time consuming. The aim of this study was to explore the limitations of estimates and develop alternative morphometric solutions for canine cutaneous mast cell tumors (ccMCT). We assessed the following nuclear evaluation methods for measurement accuracy, reproducibility, and prognostic utility: 1) anisokaryosis (karyomegaly) estimates by 11 pathologists; 2) gold standard manual morphometry of at least 100 nuclei; 3) practicable manual morphometry with stratified sampling of 12 nuclei by 9 pathologists; and 4) automated morphometry using a deep learning-based segmentation algorithm. The study dataset comprised 96 ccMCT with available outcome information. The study dataset comprised 96 ccMCT with available outcome information. Inter-rater reproducibility of karyomegaly estimates was low ($\kappa$ = 0.226), while it was good (ICC = 0.654) for practicable morphometry of the standard deviation (SD) of nuclear size. As compared to gold standard manual morphometry (AUC = 0.839, 95% CI: 0.701 - 0.977), the prognostic value (tumor-specific survival) of SDs of nuclear area for practicable manual morphometry (12 nuclei) and automated morphometry were high with an area under the ROC curve (AUC) of 0.868 (95% CI: 0.737 - 0.991) and 0.943 (95% CI: 0.889 - 0.996), respectively. This study supports the use of manual morphometry with stratified sampling of 12 nuclei and algorithmic morphometry to overcome the poor reproducibility of estimates.
BibTeX
@inproceedings{haghofer2024nuclear,
  author = {Andreas Haghofer and Eda Parlak and Alexander Bartel and Taryn A. Donovan and Charles-Antoine Assenmacher and Pompei Bolfa and Michael J. Dark and Andrea Fuchs-Baumgartinger and Andrea Klang and Kathrin Jäger and Robert Klopfleisch and Sophie Merz and Barbara Richter and F. Yvonne Schulman and Jonathan Ganz and Josef Scharinger and Marc Aubreville and Stephan M. Winkler and Matti Kiupel and Christof A. Bertram},
  title = {{Nuclear Morphometry using a Deep Learning-based Algorithm has Prognostic Relevance for Canine Cutaneous Mast Cell Tumors}},
  booktitle = {Veterinary},
  year = {2024},
  doi = {10.1177/03009858241295399},
}
Figure