RCdpia: A Renal Carcinoma Digital Pathology Image Annotation dataset based on pathologists

Sun Qing-rong, Weixiang Zhong, Jie Zhou, Chong Lai, Xiaodong Teng, Maode Lai

TL;DR

Researchers have released RCdpia, a meticulously cleaned and pathologist-annotated dataset of over 900 renal carcinoma whole-slide images (WSIs) sourced from The Cancer Genome Atlas (TCGA). By validating these annotations with ResNet models across different medical centers, the study highlights a critical "domain shift" problem: AI models trained on one hospital's data often fail at another due to variations in tissue staining and image quality.

What Problem Does This Paper Address?

Artificial Intelligence shows immense promise in digital pathology, but it faces a "garbage in, garbage out" dilemma. Public repositories like The Cancer Genome Atlas (TCGA) are goldmines of data, but they are often "noisy." Raw slides may contain "graffiti" (pathologist pen marks), air bubbles, or poor focus. Furthermore, renal cell carcinoma is highly heterogeneous—meaning the tumor looks different even within the same patient—making automated diagnosis difficult.

The authors address two main gaps:

  1. Data Quality: The lack of high-quality, expert-verified labels for tumor vs. normal regions in kidney cancer WSIs.
  2. Model Robustness: The tendency for models to perform exceptionally well on internal data but fail when deployed at a different hospital (the "multi-center" problem).

Methods

The researchers followed a multi-step pipeline to transform raw images into a robust AI training resource:

  • Expert Annotation: Two pathologists manually screened 928 WSIs from TCGA. They used color-coded digital "ink" to delineate three specific regions: the entire tumor outline (tALL), characteristic tumor regions (tumor), and adjacent normal tissues (normal).
  • Patch-Based Processing: Because whole-slide images are too massive for standard GPUs, they segmented the annotated regions into $256 \times 256$ pixel patches.
  • Deep Learning Classification: They employed ResNet-18 and ResNet-50 architectures to classify the three main subtypes of renal cancer: Kidney Clear Cell Carcinoma (KIRC), Chromophobe (KICH), and Papillary Cell Carcinoma (KIRP).
  • Cross-Center Validation: To test real-world applicability, they validated their TCGA-trained models on a completely independent dataset from the First Affiliated Hospital of Zhejiang University (FAHZU).

Datasets

The study utilizes two primary data sources:

  • RCdpia (The New Benchmark): Derived from TCGA, this dataset includes 928 cases (519 KIRC, 109 KICH, and 300 KIRP). It is now publicly accessible for the research community.
  • FAHZU Dataset: An external validation set consisting of 682 cases (373 KIRC, 142 KICH, and 167 KIRP) from a different clinical environment.

Key Insights and Ideas

  • The "Center Effect" is Real: A model that achieved ~99% accuracy on TCGA data saw a "marked decline" when first applied to the FAHZU dataset. This confirms that AI in pathology is highly sensitive to local variations in slide preparation and scanning hardware.
  • Simpler Can Be Better: Interestingly, the ResNet-18 model (a shallower architecture) achieved a higher average accuracy across the three subtypes than the more complex ResNet-50 in several validation scenarios. This suggests that for histopathology patches, deeper isn't always better; sometimes, it just leads to overfitting.
  • The Need for "Stain Normalization": The authors argue that the primary hurdle for universal pathology AI isn't the algorithm itself, but the "color space." Differences in how labs use Hematoxylin and Eosin (H&E) stains create a visual gap that AI struggles to bridge without specific normalization techniques.

Key Results

  • Internal Accuracy: On the curated TCGA testing set, the ResNet models achieved approximately 99% accuracy across the three kidney cancer subtypes.
  • External Validation: When testing on the FAHZU dataset, accuracy was initially low but improved significantly as the training set size increased.
  • Subtype Performance: As shown in Table 1, with a training set of 20 WSIs, ResNet-18 reached accuracies of 94.6% for KIRC and 92.6% for KIRP, though KICH remained slightly more challenging at 89.3%.

Limitations and Future Work

The authors acknowledge that while their dataset is a significant step forward, the "domain shift" between different hospitals remains a hurdle. They suggest that future work should focus on color space reconstruction. Rather than just training on more data, they propose developing deep learning algorithms that can standardize the appearance of nuclei and cytoplasm across all slides, regardless of which lab produced them. This would move the field closer to a "plug-and-play" AI diagnostic tool.

Summary updated

Abstract

The annotation of digital pathological slide data for renal cell carcinoma is of paramount importance for correct diagnosis of artificial intelligence models due to the heterogeneous nature of the tumor. This process not only facilitates a deeper understanding of renal cell cancer heterogeneity but also aims to minimize noise in the data for more accurate studies. To enhance the applicability of the data, two pathologists were enlisted to meticulously curate, screen, and label a kidney cancer pathology image dataset from The Cancer Genome Atlas Program (TCGA) database. Subsequently, a Resnet model was developed to validate the annotated dataset against an additional dataset from the First Affiliated Hospital of Zhejiang University. Based on these results, we have meticulously compiled the TCGA digital pathological dataset with independent labeling of tumor regions and adjacent areas (RCdpia), which includes 109 cases of kidney chromophobe cell carcinoma, 486 cases of kidney clear cell carcinoma, and 292 cases of kidney papillary cell carcinoma. This dataset is now publicly accessible at http://39.171.241.18:8888/RCdpia/. Furthermore, model analysis has revealed significant discrepancies in predictive outcomes when applying the same model to datasets from different centers. Leveraging the RCdpia, we can now develop more precise digital pathology artificial intelligence models for tasks such as normalization, classification, and segmentation. These advancements underscore the potential for more nuanced and accurate AI applications in the field of digital pathology.
BibTeX
@misc{qingrong2024rcdpia,
  author = {Sun Qing-rong and Weixiang Zhong and Jie Zhou and Chong Lai and Xiaodong Teng and Maode Lai},
  title = {{RCdpia: A Renal Carcinoma Digital Pathology Image Annotation dataset based on pathologists}},
  year = {2024},
  eprint = {2403.11211},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.11211},
}
Figure