Dataset on Bi- and Multi-Nucleated Tumor Cells in Canine Cutaneous Mast Cell Tumors

Christof A. Bertram, Taryn A. Donovan, Marco Tecilla, Florian Bartenschlager, Marco Fragoso, Frauke Wilm, Christian Marzahl, Katharina Breininger, Andreas Maier, Robert Klopfleisch, Marc Aubreville

TL;DR

This paper introduces the first open-source dataset of binucleated and multinucleated tumor cells in canine cutaneous mast cell tumors, containing over 21,000 expert-reviewed annotations. Using an algorithmic-aided labeling approach, the authors establish a strong deep learning baseline that outperforms individual veterinary pathologists on the dataset ground truth, while exposing a surprisingly high degree of inter-observer variability in manual tumor grading.

What Problem Does This Paper Address?

In veterinary oncology, canine cutaneous mast cell tumors (ccMCT) are a common skin cancer in dogs. To predict how aggressively a tumor will behave, pathologists evaluate tissue biopsies under a microscope using specific grading schemes. Key indicators of poor patient outcomes include the presence of binucleated cells (BiNC) (cells with two nuclei) and multinucleated cells (MuNC) (cells with three or more nuclei). These abnormal cells signify polyploidy—an increased amount of genetic material that helps tumors grow, progress, and resist chemotherapy.

However, manually counting these cells across an entire tissue sample is tedious and highly error-prone. Pathologists frequently miss target cells due to fatigue, and different experts often disagree on what constitutes a multi-nucleated cell versus an indented single nucleus. To develop reliable machine learning tools that can automate and standardize this task, researchers need large, high-quality labeled datasets—which, until now, did not exist for this application.

Methods

The authors developed a human-in-the-loop, algorithmic-aided labeling approach to build their dataset from 32 whole slide images (WSIs).

  1. Manual Screening: A single pathologist initially screened the images, labeling roughly 10,000 BiNCs and 775 MuNCs.
  2. AI-Assisted Expansion: To combat human omission errors, the authors split this initial data into three folds to train an object detection network. They ran this network back over the images with an intentionally low detection threshold to catch as many potential candidates as possible, accepting a high false-positive rate to minimize confirmation bias.
  3. Expert Review: The pathologist manually reviewed all 73,543 AI-generated candidates, confirming an additional 9,602 BiNCs and 641 MuNCs.

For their baseline automated detection model, the authors implemented a two-stage deep learning pipeline:

  • Stage 1 (Object Detection): A customized RetinaNet with a pre-trained ResNet-18 stem to propose candidate cells.
  • Stage 2 (Patch Classification): A secondary ResNet-18 classifier to filter out "hard negatives"—imposters like cells with heavily indented or lobulated single nuclei that look deceptively multinucleated.

Datasets

The study introduces a newly annotated benchmark dataset based on 32 high-resolution histological whole slide images (WSIs) of ccMCT (originally sourced from a prior mitosis dataset).

  • Total Annotations: 19,983 labels of BiNC and 1,416 labels of MuNC.
  • Data Split: 21 WSIs for training, 11 WSIs for testing.
  • Availability: The dataset is fully open-source and accessible as a SQLite3 database on GitHub.

Key Insights and Ideas

  • The "Invisible" Tumor Cells: Manual screening alone is highly insufficient. The AI-assisted pipeline nearly doubled the number of verified BiNC (+92.5%) and MuNC (+82.7%) labels that the pathologist initially missed.
  • High Inter-Observer Disagreement: When six independent veterinary pathologists graded the same regions of interest, their consensus was remarkably low. Many cells were flagged by only a single pathologist, highlighting how subjective manual visual evaluation truly is.
Pathologist voting consensus and visual examples of binucleated and multinucleated cells.
  • Biological Co-localisation: The density of BiNCs strongly correlated with MuNCs ($r = 0.66$), but showed much weaker correlation with active cell division (mitotic figures). This supports the biological theory that these cells primarily form via endoreplication (an incomplete cell cycle where the cell duplicates its DNA but fails to divide) rather than cell fusion.

Key Results

The deep learning model achieved highly competitive baseline results on the 11 test whole slide images, achieving an $F_1$ score of 0.675 for BiNC and 0.628 for MuNC.

When compared directly against six veterinary pathologists on specific regions of interest:

  • Against Dataset Ground Truth: The deep learning model (DL) significantly outperformed all human experts. For BiNC, the model achieved an $F_1$ of 0.667 (vs. a human median of 0.314). For MuNC, the model scored 0.685 (vs. a human median of 0.477).
  • Against Multi-Expert Consensus: When evaluating performance against strict multi-pathologist consensus (e.g., three-vote ground truth), the model's performance dropped (to ~0.48), matching the median performance of the human experts. This drop underscores that the model learned the specific style of the primary annotator, yet still performed on par with human consensus.
Performance comparison for binucleated cell detection.
Performance comparison for multinucleated cell detection.

Limitations and Future Work

The primary limitation is that the dataset's ground truth was ultimately determined by a single supervising pathologist. Given the high inter-rater variability exposed in the study, future iterations would benefit from a multi-expert consensus dataset to improve label consistency.

Additionally, the images come from a single digital scanner and staining protocol. To make the deep learning models clinically viable, the dataset needs to expand to include variations in tissue preparation and scanning hardware from multiple institutions.

Summary updated

Abstract

Tumor cells with two nuclei (binucleated cells, BiNC) or more nuclei (multinucleated cells, MuNC) indicate an increased amount of cellular genetic material which is thought to facilitate oncogenesis, tumor progression and treatment resistance. In canine cutaneous mast cell tumors (ccMCT), binucleation and multinucleation are parameters used in cytologic and histologic grading schemes (respectively) which correlate with poor patient outcome. For this study, we created the first open source data-set with 19,983 annotations of BiNC and 1,416 annotations of MuNC in 32 histological whole slide images of ccMCT. Labels were created by a pathologist and an algorithmic-aided labeling approach with expert review of each generated candidate. A state-of-the-art deep learning-based model yielded an $F_1$ score of 0.675 for BiNC and 0.623 for MuNC on 11 test whole slide images. In regions of interest ($2.37 mm^2$) extracted from these test images, 6 pathologists had an object detection performance between 0.270 - 0.526 for BiNC and 0.316 - 0.622 for MuNC, while our model archived an $F_1$ score of 0.667 for BiNC and 0.685 for MuNC. This open dataset can facilitate development of automated image analysis for this task and may thereby help to promote standardization of this facet of histologic tumor prognostication.
BibTeX
@inproceedings{bertram2021dataset,
  author = {Christof A. Bertram and Taryn A. Donovan and Marco Tecilla and Florian Bartenschlager and Marco Fragoso and Frauke Wilm and Christian Marzahl and Katharina Breininger and Andreas Maier and Robert Klopfleisch and Marc Aubreville},
  title = {{Dataset on Bi- and Multi-Nucleated Tumor Cells in Canine Cutaneous Mast Cell Tumors}},
  booktitle = {Bildverarbeitung für die Medizin},
  year = {2021},
  url = {https://openreview.net/forum?id=Yr79BMicGzp},
}
Figure