Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives

Ayhan Can Erdur, Daniel Rusche, Daniel Scholz, Johannes Kiechle, Stefan M. Fischer, Óscar Llorián-Salvador, Josef A. Buchner, Mai Q. Nguyen, Lucas Etzel, Jonas Weidner, Marie‐Christin Metz, Benedikt Wiestler, Julia A. Schnabel, Daniel Rueckert, Stephanie E. Combs, Jan C. PeekenStrahlentherapie 2024 · DOI

Abstract

The rapid development of artificial intelligence (AI) has gained importance, with many tools already entering our daily lives. The medical field of radiation oncology is also subject to this development, with AI entering all steps of the patient journey. In this review article, we summarize contemporary AI techniques and explore the clinical applications of AI-based automated segmentation models in radiotherapy planning, focusing on delineation of organs at risk (OARs), the gross tumor volume (GTV), and the clinical target volume (CTV). Emphasizing the need for precise and individualized plans, we review various commercial and freeware segmentation tools and also state-of-the-art approaches. Through our own findings and based on the literature, we demonstrate improved efficiency and consistency as well as time savings in different clinical scenarios. Despite challenges in clinical implementation such as domain shifts, the potential benefits for personalized treatment planning are substantial. The integration of mathematical tumor growth models and AI-based tumor detection further enhances the possibilities for refining target volumes. As advancements continue, the prospect of one-stop-shop segmentation and radiotherapy planning represents an exciting frontier in radiotherapy, potentially enabling fast treatment with enhanced precision and individualization.

Introduction: The Precision Bottleneck in Radiotherapy

Radiotherapy planning is a high-stakes game of geometry. Clinicians must precisely delineate the Gross Tumor Volume (GTV), the Clinical Target Volume (CTV) (which includes microscopic spread), and Organs at Risk (OARs) to ensure the radiation kills the cancer without destroying healthy tissue.

Historically, this "contouring" was done manually or via atlas-based methods (matching new patients to a database of old ones). Both are problematic: manual work is slow and varies wildly between doctors, while atlases struggle with anatomical diversity. As radiotherapy moves toward "adaptive" planning—where the plan is updated daily as the patient’s anatomy shifts—the need for near-instant, consistent, and highly accurate autosegmentation has become a critical bottleneck.

Methodological Evolution: From U-Nets to Transformers

The paper categorizes the evolution of autosegmentation through several key technical phases:

The U-Net Standard

The backbone of medical imaging AI, utilizing an encoder-decoder structure with "skip connections" to preserve spatial detail. This architecture allows the model to learn both high-level semantic features and low-level spatial information simultaneously.

Standard U-Net architecture for medical image segmentation, showing the encoder-decoder path and skip connections.

Transformer-based Architectures (UNetR)

A newer shift from standard Convolutional Neural Networks (CNNs) to Transformer-based architectures. Unlike CNNs, which look at local pixels, Transformers use "self-attention" to capture long-range dependencies across the entire 3D image volume, which is particularly useful for complex anatomical relationships.

Transformer-based architecture (UNetR) utilizing self-attention mechanisms to process 3D image patches.

Generative Models and Explainability

Beyond basic segmentation, Generative Models (GANs & Diffusion) are used to synthesize high-quality labeled training data or improve performance via adversarial loss. Furthermore, the authors highlight the move toward "White Box" AI. Techniques like Grad-CAM and Uncertainty Quantification (distinguishing between model ignorance and image noise) are essential for building clinical trust.

Key Insights: Target Volume 2.0

  • Integrating Growth Models: One of the most forward-thinking sections discusses integrating mathematical tumor growth models (like the Fisher-Kolmogorov equation) with AI. Instead of just drawing a circle around a visible lump, the AI predicts where microscopic cells are likely to have migrated based on tissue density and diffusion patterns.
  • The "Turing Test" for Contours: In some studies, blinded experts preferred AI-generated contours over manual ones, suggesting a tipping point in quality.
  • Efficiency vs. Accuracy: AI provides a "stable baseline" that reduces the cognitive load on physicians, who now act as editors rather than authors.
  • One-Stop-Shop Radiotherapy: The authors envision a future where a patient is scanned, segmented, planned, and treated in a single session, drastically reducing the time-to-treatment.

Quantitative Performance and Clinical Impact

  • Time Savings: In lung cancer (NSCLC) cases, deep learning models saved a median of 10 minutes per patient compared to manual contouring. For thoracic OARs, editing time was reduced to just 7.5 minutes.
  • Accuracy: Many commercial U-Net-based tools now achieve Dice Similarity Coefficients (DSC) > 0.85 for major organs, performing at or above the level of atlas-based methods.
  • Speed: Some models (like Mask R-CNN for rectal cancer) can process a case in 1.3 seconds, compared to 200 seconds for a human radiologist.

Challenges and Future Horizons

Despite the potential, several challenges remain:

  • Domain Shift: Models trained on specific scanner types or protocols may fail when applied to different clinical environments.
  • Small/Mobile Organs: AI still struggles with very small structures (like the optic nerve) or organs that change shape rapidly (like the stomach).
  • Human-in-the-Loop: For the foreseeable future, AI is an assistant. The authors emphasize that physician review is non-negotiable, particularly for complex target volumes where clinical judgment is paramount.

Summary updated

Abstract

The rapid development of artificial intelligence (AI) has gained importance, with many tools already entering our daily lives. The medical field of radiation oncology is also subject to this development, with AI entering all steps of the patient journey. In this review article, we summarize contemporary AI techniques and explore the clinical applications of AI-based automated segmentation models in radiotherapy planning, focusing on delineation of organs at risk (OARs), the gross tumor volume (GTV), and the clinical target volume (CTV). Emphasizing the need for precise and individualized plans, we review various commercial and freeware segmentation tools and also state-of-the-art approaches. Through our own findings and based on the literature, we demonstrate improved efficiency and consistency as well as time savings in different clinical scenarios. Despite challenges in clinical implementation such as domain shifts, the potential benefits for personalized treatment planning are substantial. The integration of mathematical tumor growth models and AI-based tumor detection further enhances the possibilities for refining target volumes. As advancements continue, the prospect of one-stop-shop segmentation and radiotherapy planning represents an exciting frontier in radiotherapy, potentially enabling fast treatment with enhanced precision and individualization.
BibTeX
@inproceedings{erdur2024deep,
  author = {Ayhan Can Erdur and Daniel Rusche and Daniel Scholz and Johannes Kiechle and Stefan M. Fischer and Óscar Llorián-Salvador and Josef A. Buchner and Mai Q. Nguyen and Lucas Etzel and Jonas Weidner and Marie‐Christin Metz and Benedikt Wiestler and Julia A. Schnabel and Daniel Rueckert and Stephanie E. Combs and Jan C. Peeken},
  title = {{Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives}},
  booktitle = {Strahlentherapie},
  year = {2024},
  doi = {10.1007/s00066-024-02262-2},
}
Figure