A whole-body FDG-PET/CT Dataset with manually annotated Tumor Lesions

Sergios Gatidis, Tobias Hepp, Marcel Früh, Christian la Fougère, Konstantin Nikolaou, Christina Pfannenberg, Bernhard Schölkopf, Thomas Küstner, Clemens C. Cyran, Daniel L. RubinScientific 2022 · DOI

Abstract

This paper presents a large-scale, publicly available dataset of 1,014 whole-body Fluorodeoxyglucose (FDG)-PET/CT studies designed to advance deep learning-based automated cancer detection. The dataset includes 501 studies of patients with malignant lymphoma, melanoma, and non-small cell lung cancer (NSCLC), alongside 513 negative control studies. All malignant lesions were manually segmented in 3D by expert radiologists. We provide anonymized DICOM files, segmentation masks, and processing scripts, demonstrating the dataset's utility through a baseline deep learning model that achieves high correlation with human expert labels.

Introduction

While deep learning has revolutionized medical imaging, its progress in PET/CT (Positron Emission Tomography/Computed Tomography) has been hampered by a significant lack of high-quality, annotated data. PET/CT is a hybrid modality where CT provides anatomical detail and PET reveals metabolic activity. A major challenge for automated systems is distinguishing malignant tumors from healthy organs—such as the brain, heart, and bladder—that naturally exhibit high metabolic activity. Without large-scale, manually annotated datasets, developing reliable algorithms for automated tumor burden measurement remains difficult.

Methodology

To address this "data desert," we curated a standardized resource of 1,014 studies acquired on a single Siemens Biograph mCT scanner between 2014 and 2018. This consistency eliminates inter-scanner variability, providing a clean benchmark for algorithm development.

Expert Annotation

An experienced radiologist with over 10 years of expertise manually segmented every FDG-avid malignant lesion identified in the clinical reports. These segmentations were performed in a slice-by-slice (3D) manner on the PET images to ensure high-fidelity ground truth labels.

Example of a manual 3D segmentation mask (green) applied to a PET-positive malignant lesion.

Baseline Model Training

To validate the dataset, we trained a baseline model using the nnU-Net framework, a state-of-the-art 3D U-Net architecture for medical image segmentation. We employed 5-fold cross-validation to assess the model's ability to replicate expert human annotations.

Dataset Characteristics

The FDG-PET-CT-Lesions dataset is hosted on The Cancer Imaging Archive (TCIA) and includes:

  • Positive Samples (501): Patients with histologically confirmed malignant lymphoma, melanoma, or NSCLC.
  • Negative Controls (513): Patients without metabolically active malignant disease, essential for training models to avoid false positives in healthy tissues.
  • Accessibility: Data is provided in original DICOM format with scripts for conversion to NIfTI, mha, or hdf5 formats.

Key Results

The baseline AI model demonstrated the high quality of the manual annotations through several key metrics:

  • Volume Correlation: A high correlation (r = 0.85) was found between the AI’s predicted tumor volume and the manual expert volume.
  • Segmentation Accuracy: The model achieved a mean Dice score of 0.73 on positive datasets.
  • Clinical Robustness: The mean false positive volume was limited to 8.1 ml, indicating the model effectively ignored physiological "hot spots."
Histograms showing the distribution of metabolic parameters (mean SUV, MTV, and TLG) within the dataset.

Discussion and Future Work

The primary limitation of this dataset is its single-center, single-scanner origin. While this ensures consistency, further validation on multi-vendor data is necessary for broader clinical application. Additionally, while the current focus is on three specific cancer types, the dataset provides a foundation for future expansion into other malignancies and longitudinal studies to track treatment response. This resource fills a critical gap, providing the high-quality labels needed to train the next generation of automated cancer monitoring tools.

Summary updated

Abstract

We describe a publicly available dataset of annotated Positron Emission Tomography/Computed Tomography (PET/CT) studies. 1014 whole body Fluorodeoxyglucose (FDG)-PET/CT datasets (501 studies of patients with malignant lymphoma, melanoma and non small cell lung cancer (NSCLC) and 513 studies without PET-positive malignant lesions (negative controls)) acquired between 2014 and 2018 were included. All examinations were acquired on a single, state-of-the-art PET/CT scanner. The imaging protocol consisted of a whole-body FDG-PET acquisition and a corresponding diagnostic CT scan. All FDG-avid lesions identified as malignant based on the clinical PET/CT report were manually segmented on PET images in a slice-per-slice (3D) manner. We provide the anonymized original DICOM files of all studies as well as the corresponding DICOM segmentation masks. In addition, we provide scripts for image processing and conversion to different file formats (NIfTI, mha, hdf5). Primary diagnosis, age and sex are provided as non-imaging information. We demonstrate how this dataset can be used for deep learning-based automated analysis of PET/CT data and provide the trained deep learning model.
BibTeX
@inproceedings{gatidis2022whole,
  author = {Sergios Gatidis and Tobias Hepp and Marcel Früh and Christian la Fougère and Konstantin Nikolaou and Christina Pfannenberg and Bernhard Schölkopf and Thomas Küstner and Clemens C. Cyran and Daniel L. Rubin},
  title = {{A whole-body FDG-PET/CT Dataset with manually annotated Tumor Lesions}},
  booktitle = {Scientific},
  year = {2022},
  doi = {10.1038/s41597-022-01718-3},
}
Figure