Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types

Le Hou, Rajarsi Gupta, John S. Van Arnam, Yuwei Zhang, Kaustubh Sivalenka, Dimitris Samaras, Tahsin Kurç, Joel Saltz

Abstract

The distribution and appearance of nuclei are essential markers for the diagnosis and study of cancer. Despite the importance of nuclear morphology, there is a lack of large scale, accurate, publicly accessible nucleus segmentation data. To address this, we developed an analysis pipeline that segments nuclei in whole slide tissue images from multiple cancer types with a quality control process. We have generated nucleus segmentation results in 5,060 Whole Slide Tissue images from 10 cancer types in The Cancer Genome Atlas (TCGA). One key component of our work is that we carried out a multi-level quality control process (WSI-level and image patch-level), to evaluate the quality of our segmentation results. The image patch-level quality control used manual segmentation ground truth data from 1,356 sampled image patches. The datasets we publish in this work consist of roughly 5 billion quality controlled nuclei from more than 5,060 TCGA WSIs from 10 different TCGA cancer types and 1,356 manually segmented TCGA image patches from the same 10 cancer types plus additional 4 cancer types.

Introduction

In cancer pathology, the shape, size, and distribution of cell nuclei are vital biomarkers for diagnosis and predicting patient outcomes. However, analyzing these features manually is impossible at scale; a single tissue slide can contain over a million nuclei. While deep learning can automate this, existing models often struggle to generalize across different cancer types due to variations in tissue texture or staining. Furthermore, training these models requires massive amounts of manually annotated data, which is expensive and slow to produce. Prior to this work, public datasets were limited to thousands or a few hundred thousand nuclei. This research introduces a dataset of roughly 5 billion segmented nuclei, providing a critical foundation for large-scale computational pathology.

Methods: Synthetic Data and Efficient Inference

The researchers developed a robust analysis pipeline centered on a U-Net convolutional neural network (CNN). A primary innovation was the use of a GAN-based (Generative Adversarial Network) "texture inpainting" module to overcome the manual annotation bottleneck.

The texture inpainting module generates realistic synthetic pathology patches by combining background textures with modeled chromatin clearing and random polygons to simulate nuclei.

This module allowed the creation of 500,000 realistic, synthetic pathology patches where the "ground truth" location of every nucleus was known perfectly by design. The U-Net was then trained on a hybrid dataset of these synthetic patches and three smaller, high-quality real-world datasets. This approach forced the model to learn robust features that generalize across different organs. Additionally, the U-Net architecture proved to be roughly 10x faster than more complex models like Mask R-CNN, requiring only 3 GPU hours per whole slide image (WSI) compared to 30, making the processing of over 5,000 slides feasible.

Multi-level Quality Control

To ensure the reliability of the 5 billion segmented nuclei, a two-stage quality control (QC) process was implemented. At the WSI-level, statistical outliers—slides with anomalous nuclei counts or sizes—were flagged for visual inspection. At the patch-level, the team manually segmented 1,356 patches to quantitatively validate the model’s accuracy.

Comparison of manual segmentation methods: the second column shows the "human-in-the-loop" approach of correcting Mask R-CNN results, which was faster than labeling from scratch (columns 3-5).

To create this manual ground truth efficiently, graduate students corrected Mask R-CNN outputs rather than drawing every boundary from scratch. This "human-in-the-loop" strategy significantly reduced labor while maintaining high accuracy. The QC process identified 576 slides with suboptimal results, which are explicitly flagged in the final dataset to guide downstream researchers.

Key Results and Impact

The study successfully segmented nuclei across 5,060 WSIs from 10 cancer types (including BRCA, LUAD, and GBM). Across all types, the automatic segmentation achieved a Dice coefficient of at least 77% and an instance-level Dice of at least 62% compared to manual labels. The correlation between the model's nuclei count and human counts was remarkably high (Pearson correlation = 0.932).

This dataset represents a massive scaling of available resources for the scientific community. By proving that synthetic data can effectively train models for diverse cancer types, the authors have provided a blueprint for future large-scale histopathology studies.

Limitations and Future Directions

Despite its scale, the pipeline has limitations. Images were processed in 4,000 x 4,000-pixel tiles due to memory constraints, which can occasionally split nuclei sitting on tile borders. Furthermore, the inherent inter-observer variability in pathology means that even human experts only agree on "ground truth" about 75-80% of the time. Future work will involve using this dataset to train more advanced models and conducting pan-cancer studies to link nuclear morphology directly to genetic mutations and clinical outcomes.

Summary updated

Abstract

The distribution and appearance of nuclei are essential markers for the diagnosis and study of cancer. Despite the importance of nuclear morphology, there is a lack of large scale, accurate, publicly accessible nucleus segmentation data. To address this, we developed an analysis pipeline that segments nuclei in whole slide tissue images from multiple cancer types with a quality control process. We have generated nucleus segmentation results in 5,060 Whole Slide Tissue images from 10 cancer types in The Cancer Genome Atlas. One key component of our work is that we carried out a multi-level quality control process (WSI-level and image patch-level), to evaluate the quality of our segmentation results. The image patch-level quality control used manual segmentation ground truth data from 1,356 sampled image patches. The datasets we publish in this work consist of roughly 5 billion quality controlled nuclei from more than 5,060 TCGA WSIs from 10 different TCGA cancer types and 1,356 manually segmented TCGA image patches from the same 10 cancer types plus additional 4 cancer types.
BibTeX
@inproceedings{hou2020dataset,
  author = {Le Hou and Rajarsi Gupta and John S. Van Arnam and Yuwei Zhang and Kaustubh Sivalenka and Dimitris Samaras and Tahsin Kurç and Joel Saltz},
  title = {{Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types}},
  booktitle = {Scientific},
  year = {2020},
  doi = {10.1038/s41597-020-0528-1},
}
Figure