The autoPET3 Challenge: Automated Lesion Segmentation in Whole-Body PET/CT $\unicode{x2013}$ Multitracer Multicenter Generalization

Jakob Dexl, Katharina Jeblick, Andreas Mittermeier, Balthasar Schachtner, Anna Theresa Stüber, Johanna Topalis, Maximilian Rokuss, Fabian Isensee, Klaus H. Maier-Hein, Hamza Kalisch, Jens Kleesiek, Constantin M. Seibold, Hussain Alasmawi, Lap Yan Lennon Chan, Yixuan Yuan, Alexander Jaus, Rainer Stiefelhagen, Pauline Ornela Megne Choudja, Konstantin Nikolaou, Christian La Fougère, Sergios Gatidis, Matthias P. Fabritius, Maurice Heimer, Gizem Abaci, Lalith Kumar Shiyam Sundar, Rudolf A. Werner, Jens Ricke, Clemens C. Cyran, Thomas Küstner, Michael IngrischarXiv 2026 · arXiv

TL;DR

The autoPET3 challenge benchmarks automated lesion segmentation in whole-body PET/CT scans, specifically testing how well models handle "compositional generalization"—unseen combinations of radiotracers and clinical centers. While top-tier deep learning models excel on familiar data domains, generalizing to new tracer-center pairings remains an open problem characterized by systematic volume overestimation. Crucially, statistical analysis reveals that patient-level case difficulty impacts segmentation performance far more than the specific choice of deep learning architecture among top teams.

What Problem Does This Paper Address?

Whole-body PET/CT imaging is a cornerstone of modern oncology, vital for cancer diagnosis, staging, and treatment monitoring. However, manually delineating tumor volumes is a tedious, labor-intensive task that can take radiologists up to 90 minutes per patient. While automated deep learning models offer a solution, they frequently stumble when deployed across different hospitals due to "distribution shifts"—variations in scanner hardware, reconstruction protocols, and patient demographics.

The autoPET3 challenge introduces a more complex hurdle: compositional generalization. Instead of just testing models on a completely new hospital, the challenge entangles two different radiotracers ($[^{18}\text{F}]$-FDG and $[^{18}\text{F}]/[^{68}\text{Ga}]$-PSMA) across two clinical centers. Models are trained on specific tracer-center pairs (e.g., FDG at Center A, PSMA at Center B) and must generalize to unseen combinations (e.g., PSMA at Center A). This requires networks to robustly differentiate true pathological lesions from tracer-specific physiological background uptake (such as brain activity in FDG or salivary gland uptake in PSMA) under unfamiliar imaging conditions.

Fig. 1. Representative test cases illustrating the key challenges for automated lesion segmentation: substantial inter-patient heterogeneity in disease extent, from a single lesion to a high metastatic burden, as well as tracer-specific phys- iological uptake patterns. Cases include PSMA (left) and FDG (right) scans from the LMU (top) and UKT (bottom) cohorts. Red overlays denote manual segmentations; intensity reflects tracer uptake.

Methods

The challenge featured two distinct submission tracks:

  1. Award Category 1 (AC1) – Open Architecture: Participants had complete freedom over model architectures, training configurations, and external datasets.
  2. Award Category 2 (AC2) – Data-Centric Excellence: Participants were restricted to a fixed baseline model architecture and could only modify the data pipeline (e.g., preprocessing, augmentations, synthetic data generation).

Out of 27 submitted algorithms from 17 teams, the vast majority relied on the 3D nnU-Net framework, concatenating PET and CT volumes as multi-channel inputs. To handle the multi-tracer data, two primary paradigms emerged:

  • Global Models: A single network trained to segment lesions across both tracers simultaneously.
  • Two-Stream Routing: A lightweight classifier first identifies the tracer type from Maximum Intensity Projections (MIPs), then routes the scan to a tracer-specific expert network.

The winning algorithm (LesionTracer A) utilized a global model with an upscaled backbone (ResEnc L), dual heads for simultaneous lesion and organ segmentation, and a novel data augmentation technique designed to simulate PET/CT registration errors.

Datasets

The training data combined two massive, publicly available datasets:

  • 1,014 $[^{18}\text{F}]$-FDG PET/CT studies from the University Hospital Tübingen (UKT).
  • 597 $[^{18}\text{F}]/[^{68}\text{Ga}]$-PSMA PET/CT studies from the LMU University Hospital Munich (constituting the largest public annotated PSMA dataset to date).

The held-out test set consisted of 200 studies, evenly split (50 each) across four conditions: the two familiar training combinations (FDG-UKT, PSMA-LMU) and two completely unseen compositional pairs (PSMA-UKT, FDG-LMU).

Key Insights and Ideas

  • The "Patient Effect" Dominates: A linear mixed-effects analysis revealed that patient heterogeneity and case difficulty account for 61% of the unexplained variance in performance. In contrast, algorithmic differences between the top teams accounted for a mere 1.3%. This means which patient case you feed the model matters vastly more than which top-tier model you choose.
  • In-Domain is Approaching Human Agreement: For familiar tracer-center combinations, top models achieved highly accurate, clinically relevant segmentations that performed on par with or even exceeded a second human reader.
  • Generalization Triggers Oversegmentation: When faced with unseen tracer-center combinations, models systematically overestimate lesion volumes. For instance, models frequently flagged the lacrimal glands as tumors in the unseen PSMA-UKT dataset because those anatomical regions were cropped out or anonymized in the training data.
  • Ensembling Tames False Positives: A post-hoc majority-vote ensemble of the top five algorithms drastically reduced false positive volumes while keeping detection sensitivity high. This indicates that different top-tier models make spatially diverse mistakes, making ensembling an effective tool for error correction.

Key Results

  • The top-ranked algorithm (LesionTracer A) achieved a mean Dice Similarity Coefficient (DSC) of 0.66, a False Negative Volume (FNV) of 3.18 mL, and a False Positive Volume (FPV) of 2.78 mL across all test conditions.
  • This represented an 8% improvement in DSC and a 5 mL reduction in false-negative volume compared to the out-of-the-box nnU-Net baseline.
  • In the data-centric track (AC2), performance gains were modest, demonstrating that data-handling innovations alone on a constrained baseline model struggle to match the impact of scaling up model capacity.
Fig. 3. Performance of all 29 submitted algorithms across the four test conditions, ordered left to right by final leaderboard position. Each column shows the results for one algorithm as as a boxplot; rows show the DSC (top), FNV (middle), and FPV (bottom). FPV and FNV are displayed on a symmetric logarithmic scale. Individual test cases are shown as points, colored by domain. Stars indicate the mean across all data points. Blue medals correspond to the winning teams in AC1, red ones to those in AC2 and anchors indicate the baselines. The right side shows the distribution of team averages per-domain.

Limitations and Future Work

The authors acknowledged several constraints in the challenge design. First, a strict 5-minute per-case runtime limit restricted teams from deploying heavier ensembles or extensive test-time augmentations. Second, because tracer types and centers were entangled, it remains difficult to isolate whether a model failed due to a center's scanner hardware or its specific annotation style. Finally, ground-truth labels contained inherent noise, particularly in high-tumor-burden cases where exhaustive manual labeling is practically impossible.

Moving forward, the organizers state that simply running another challenge with standard nnU-Net variants will not advance the field. Instead, the next iteration (autoPET IV) will pivot toward interactive segmentation, allowing clinicians to efficiently guide models out of obvious failure modes. They also emphasize the need to expand the dataset to newer tracers, such as $[^{18}\text{F}]$-FAPI and $[^{68}\text{Ga}]$-DOTATATE.

Summary updated

Abstract

We report the design and results of the third autoPET challenge (MICCAI 2024), which benchmarked automated lesion segmentation in whole-body PET/CT under a compositional generalization setting. Training data comprised 1,014 [18F]-FDG PET/CT studies from the University Hospital Tübingen and 597 [18F]/[68Ga]-PSMA PET/CT studies from the LMU University Hospital Munich, constituting the largest publicly available annotated PSMA PET/CT dataset to date. The held-out test set of 200 studies covered four tracer-center combinations, two of which represented unseen compositional pairings. A complementary data-centric award category isolated the contribution of data handling strategies by restricting participants to a fixed baseline model. Seventeen teams submitted 27 algorithms, predominantly nnU-Net-based 3D networks with PET/CT channel concatenation. The top-ranked algorithm achieved a mean DSC of 0.66, FNV of 3.18 mL, and FPV of 2.78 mL across all four test conditions, improving DSC by 8% and reducing the false-negative volume by 5 mL relative to the provided baseline. Ranking was stable across bootstrap resampling and alternative ranking schemes for the top tier. Beyond the benchmark, we provide an in-depth analysis of segmentation performance at the patient and lesion level. Three main conclusions can be drawn: (1) in-domain multitracer PET/CT segmentation is sufficient and probably approaching reader agreement; (2) compositional generalization to unseen tracer-center combinations remains an open problem mainly driven by systematic volume overestimation; (3) heterogeneity and case difficulty drive performance variation substantially more than the choice of algorithm among top-ranked teams.
BibTeX
@misc{dexl2026autopet,
  author = {Jakob Dexl and Katharina Jeblick and Andreas Mittermeier and Balthasar Schachtner and Anna Theresa Stüber and Johanna Topalis and Maximilian Rokuss and Fabian Isensee and Klaus H. Maier-Hein and Hamza Kalisch and Jens Kleesiek and Constantin M. Seibold and Hussain Alasmawi and Lap Yan Lennon Chan and Yixuan Yuan and Alexander Jaus and Rainer Stiefelhagen and Pauline Ornela Megne Choudja and Konstantin Nikolaou and Christian La Fougère and Sergios Gatidis and Matthias P. Fabritius and Maurice Heimer and Gizem Abaci and Lalith Kumar Shiyam Sundar and Rudolf A. Werner and Jens Ricke and Clemens C. Cyran and Thomas Küstner and Michael Ingrisch},
  title = {{The autoPET3 Challenge: Automated Lesion Segmentation in Whole-Body PET/CT $\textbackslash\{\}unicode\{x2013\}$ Multitracer Multicenter Generalization}},
  year = {2026},
  eprint = {2605.05775},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.05775},
}
Figure