From FDG to PSMA: A Hitchhiker's Guide to Multitracer, Multicenter Lesion Segmentation in PET/CT Imaging

Maximilian R. Rokuss, Balint Kovacs, Yannick Kirchhoff, Shuhan Xiao, Constantin Ulrich, Klaus H. Maier-Hein, Fabian IsenseearXiv 2024 · OpenReview

Abstract

Automated lesion segmentation in PET/CT scans is crucial for improving clinical workflows and advancing cancer diagnostics. However, the task is challenging due to physiological variability, different tracers used in PET imaging, and diverse imaging protocols across medical centers. To address this, the autoPET series was created to challenge researchers to develop algorithms that generalize across diverse PET/CT environments. This paper presents our solution for the autoPET III challenge, targeting multitracer, multicenter generalization using the nnU-Net framework with the ResEncL architecture. Key techniques include misalignment data augmentation and multi-modal pretraining across CT, MR, and PET datasets to provide an initial anatomical understanding. We incorporate organ supervision as a multitask approach, enabling the model to distinguish between physiological uptake and tracer-specific patterns, which is particularly beneficial in cases where no lesions are present. Compared to the default nnU-Net, which achieved a Dice score of 57.61, or the larger ResEncL (65.31) our model significantly improved performance with a Dice score of 68.40, alongside a reduction in false positive (FPvol: 7.82) and false negative (FNvol: 10.35) volumes. These results underscore the effectiveness of combining advanced network design, augmentation, pretraining, and multitask learning for PET/CT lesion segmentation. After evaluation on the test set, our approach was awarded the first place in the model-centric category (Team LesionTracer). Code is publicly available at https://github.com/MIC-DKFZ/autopet-3-submission.

Introduction

Positron Emission Tomography (PET) combined with Computed Tomography (CT) is a cornerstone of modern oncology. However, manual lesion segmentation is time-consuming and prone to inter-observer variability. Automating this process is hindered by three primary challenges: tracer variability (e.g., FDG vs. PSMA), physiological uptake in healthy organs (like the brain or bladder), and domain shifts between imaging centers. The autoPET III challenge specifically addresses these issues by requiring models to generalize across tracers and centers without prior knowledge of the tracer type.

Methodology

Our winning approach, developed by Team LesionTracer, leverages the nnU-Net framework with a Residual Encoder Large (ResEncL) architecture. The pipeline incorporates several critical innovations designed to enhance anatomical awareness and robustness.

Visualization of PET/CT input, organ supervision labels, and target lesion annotations.

Key Innovations

  1. Organ Supervision (Multitask Learning): To help the model distinguish between pathological lesions and normal physiological tracer uptake, we implemented a multitask learning strategy. The model simultaneously predicts lesion masks and segments 13 key organs. Organ labels were generated using the TotalSegmentator tool to provide "silver standard" supervision.
  2. Multi-modal Pretraining: We utilized a "MultiTalent" foundation model approach, pretraining on a massive collection of CT, MRI, and PET datasets. This provided the network with a robust initial understanding of human anatomy before fine-tuning on the challenge data.
  3. Misalignment Data Augmentation (misalDA): Recognizing that PET and CT scans are often imperfectly aligned in clinical practice, we introduced random spatial shifts between modalities during training. This teaches the model to remain robust against registration errors.
  4. Optimized Training Configuration: We employed a large patch size ($192 \times 192 \times 192$) to maximize anatomical context and removed the smoothing term from the Dice loss to improve training stability.

Results

The proposed pipeline demonstrated significant improvements over baseline models. The ablation study highlights the additive benefits of each component, particularly the combination of pretraining and organ supervision.

Ablation study showing the performance impact of different architectural and training settings on Dice scores and error volumes.

The final model achieved a global Dice score of 68.40, a substantial increase from the default nnU-Net's 57.61. Notably, the false positive volume (FPvol) was reduced by over 50% (from 19.32 to 7.82), indicating a much lower rate of "false alarms" in healthy tissues. This performance secured 1st place in the model-centric category of the autoPET III challenge.

Discussion and Conclusion

Our results suggest that "anatomy is the best teacher" for PET/CT segmentation. By forcing the model to understand the spatial context of healthy organs, it becomes significantly more adept at identifying tracer-specific patterns and avoiding false positives. Furthermore, simulating real-world imperfections through misalignment augmentation proved more effective than relying on perfectly curated data.

Future work will focus on optimizing inference times to allow for more extensive Test-Time Augmentation (TTA) and refining the balance between organ and lesion loss weights to further minimize false positives without sacrificing sensitivity.

Summary updated

Abstract

Automated lesion segmentation in PET/CT scans is crucial for improving clinical workflows and advancing cancer diagnostics. However, the task is challenging due to physiological variability, different tracers used in PET imaging, and diverse imaging protocols across medical centers. To address this, the autoPET series was created to challenge researchers to develop algorithms that generalize across diverse PET/CT environments. This paper presents our solution for the autoPET III challenge, targeting multitracer, multicenter generalization using the nnU-Net framework with the ResEncL architecture. Key techniques include misalignment data augmentation and multi-modal pretraining across CT, MR, and PET datasets to provide an initial anatomical understanding. We incorporate organ supervision as a multitask approach, enabling the model to distinguish between physiological uptake and tracer-specific patterns, which is particularly beneficial in cases where no lesions are present. Compared to the default nnU-Net, which achieved a Dice score of 57.61, or the larger ResEncL (65.31) our model significantly improved performance with a Dice score of 68.40, alongside a reduction in false positive (FPvol: 7.82) and false negative (FNvol: 10.35) volumes. These results underscore the effectiveness of combining advanced network design, augmentation, pretraining, and multitask learning for PET/CT lesion segmentation. After evaluation on the test set, our approach was awarded the first place in the model-centric category (Team LesionTracer). Code is publicly available at https://github.com/MIC-DKFZ/autopet-3-submission.
BibTeX
@misc{rokuss2024from,
  author = {Maximilian R. Rokuss and Balint Kovacs and Yannick Kirchhoff and Shuhan Xiao and Constantin Ulrich and Klaus H. Maier-Hein and Fabian Isensee},
  title = {{From FDG to PSMA: A Hitchhiker's Guide to Multitracer, Multicenter Lesion Segmentation in PET/CT Imaging}},
  year = {2024},
  eprint = {2409.09478},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.09478},
}
Figure