Deep Learning for CT Synthesis in Radiotherapy

Yemin Guo, Yi Luo, Hamed Hooshangnejad, Rui Zhang, Xue Feng, Quan Chen, Wilfred Ngwa, Kai DingBioengineering 2025 · DOI

TL;DR

This review explores the exciting advancements in using deep learning (DL) to generate synthetic Computed Tomography (sCT) images for radiotherapy. By synthesizing CT scans from other imaging modalities like MRI and CBCT, DL can significantly improve treatment planning, reduce radiation exposure, and accelerate workflows, ultimately leading to better patient outcomes.

What Problem Does This Paper Address?

Radiotherapy relies heavily on Computed Tomography (CT) scans for accurate dose calculation and treatment planning. However, CT imaging comes with drawbacks: it exposes patients to ionizing radiation, adds complexity and delays to clinical workflows, and can be challenging to acquire in real-time for adaptive treatments. Magnetic Resonance Imaging (MRI) offers superior soft-tissue contrast, crucial for precise tumor and organ delineation, but lacks the electron density information needed for dose calculation. Cone-Beam CT (CBCT), used for image-guided adaptive radiotherapy (IGART), often suffers from artifacts and low contrast, limiting its utility. To overcome these limitations, researchers are developing methods to generate synthetic CT (sCT) images from MRI or CBCT, aiming to retain the benefits of these modalities while providing the necessary information for accurate radiotherapy.

Methods

The paper provides a comprehensive overview of deep learning models used for sCT synthesis, categorizing them into four main groups:

  • Convolutional Neural Networks (CNNs): Particularly U-Net architectures and their variants, which excel at capturing local features and preserving spatial resolution through their encoder-decoder structure and skip connections.
  • Generative Adversarial Networks (GANs): Including conditional GANs (cGANs) and CycleGANs, which are adept at generating perceptually realistic images and can perform modality-to-modality translation, reducing reliance on perfectly aligned paired data.
  • Transformer-based Architectures: Leveraging self-attention mechanisms to capture long-range dependencies and global anatomical context, showing promise in improving structural integrity.
  • Diffusion Models: A newer class of models that iteratively generate images by reversing a noise process, demonstrating strong generative capabilities and improving image fidelity and anatomical consistency.

The review also discusses hybrid models that combine elements from these architectures to leverage their respective strengths. Furthermore, it delves into various training strategies, including different input data representations (2D, 2.5D, 3D), supervision paradigms (paired vs. unpaired data), and learning paradigms (loss functions, guided training, federated learning).

Datasets

Several publicly available datasets are highlighted for benchmarking sCT generation methods:

  • SynthRAD2023 and SynthRAD2025: Large-scale, multi-institutional datasets providing paired MRI-CT and CBCT-CT images across various anatomical regions, crucial for developing generalizable models.
  • Gold Atlas: Contains MRI and CT images from patients with prostate or rectal cancer, useful for MRI-to-CT synthesis.
  • Pelvic Reference Data: Includes pelvic CBCT-CT pairs with annotated landmarks, valuable for registration benchmarking and CBCT-to-CT synthesis.
  • Pancreatic-CT-CBCT-SEG dataset: Features CT and CBCT pairs from pancreatic cancer patients, notable for its challenging artifacts due to breath-hold techniques.
  • 4D Lung dataset: Provides 4D fan-beam CT and 4D CBCT scans for synthetic 4D CT generation.

The paper emphasizes the importance of these datasets for reproducibility and cross-study comparison.

Key Insights and Ideas

  • DL is Revolutionizing sCT Synthesis: Deep learning has moved sCT generation from methods requiring careful tuning and sensitive to input variations to faster, more accurate, and less intervention-dependent approaches.
  • Modality-Specific Challenges: While MRI offers excellent soft-tissue contrast, CBCT faces issues with artifacts. DL models are being tailored to address these specific limitations.
  • The Rise of Transformers and Diffusion Models: These newer architectures are showing significant promise in capturing global context and improving image fidelity, pushing the boundaries of sCT quality.
  • Unpaired Data is Key for Scalability: The difficulty in acquiring perfectly paired datasets is driving research into unpaired and unsupervised learning methods, making sCT generation more adaptable to different clinical settings.
  • Clinical Translation is the Next Frontier: While DL models achieve impressive technical results, bridging the gap to real-world clinical adoption requires addressing challenges in generalizability, standardization, and efficiency.

Key Results

The review highlights that DL-based sCT synthesis has demonstrated:

  • Improved Image Quality: Models can effectively reduce artifacts and enhance soft-tissue contrast in CBCT images.
  • Accurate Dose Calculation: sCTs have been shown to produce dose calculations that closely match those from reference CTs, supporting their clinical utility.
  • Workflow Efficiency: Simulation-free workflows, enabled by sCT, can significantly reduce the time from diagnosis to treatment initiation, which is critical for fast-progressing diseases.
  • Comparable or Superior Performance: In many studies, DL-based sCTs have achieved comparable or even superior dosimetric accuracy and anatomical fidelity compared to traditional methods.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Generalizability: Current DL models are often trained under specific, narrow conditions (e.g., single anatomical region, patient population). Future work needs to focus on developing models that generalize well across different institutions, scanners, and anatomical sites.
  • Data Challenges: The scarcity of high-quality, paired datasets remains a significant hurdle. Continued development of unpaired and unsupervised learning methods is crucial.
  • Evaluation Metrics: While intensity-based metrics are common, there's a need for more clinically interpretable metrics that directly assess dosimetric impact and anatomical accuracy in critical structures. Standardized evaluation frameworks are essential.
  • Clinical Integration: sCT synthesis is just one step in the radiotherapy workflow. Future research should focus on developing end-to-end clinical pipelines that integrate sCT generation with downstream tasks like OAR delineation and treatment planning.
  • Vendor Collaboration: Seamless integration into clinical workflows requires collaboration with treatment planning system vendors to optimize models for runtime efficiency and system compatibility.
  • Open-Source Initiatives: Promoting open-source availability of codes and models, along with standardized pipelines and benchmarks, is vital for reproducibility and accelerating clinical translation.
  • Multimodal Learning: Incorporating additional imaging modalities like PET scans could further enhance the capabilities and coverage of sCT generation.

Summary updated

Abstract

With the rapid development of artificial intelligence (AI), various deep learning (DL) methods have been introduced into radiation oncology. Among them, the generation of synthetic Computed Tomography (sCT) images has attracted increasing attention, as it supports different clinical scenarios, from image-guided adaptive radiotherapy (IGART) to the simulation-free workflow. This review provides a comprehensive overview of recent studies on DL-based sCT synthesis in radiotherapy from multiple imaging modalities, including Cone-Beam CT (CBCT), Magnetic Resonance Imaging (MRI), and diagnostic CT, and discusses their clinical applications in CBCT-based online adaptive radiotherapy, MRI-guided radiotherapy, and simulation-free workflows. We also examine the architectures of representative DL models such as convolutional neural networks (CNNs) and generative adversarial networks (GANs) and summarize emerging training strategies. Finally, we discuss current challenges of clinical translation of DL algorithms into clinical practice and suggest potential directions for future research. Overall, this paper highlights the potential of AI-driven sCT generation to advance treatment planning by reducing imaging burden, improving dose accuracy, and accelerating workflow efficiency, thus ultimately improving the treatment outcome of patient care.
BibTeX
@inproceedings{guo2025deep,
  author = {Yemin Guo and Yi Luo and Hamed Hooshangnejad and Rui Zhang and Xue Feng and Quan Chen and Wilfred Ngwa and Kai Ding},
  title = {{Deep Learning for CT Synthesis in Radiotherapy}},
  booktitle = {Bioengineering},
  year = {2025},
  doi = {10.3390/bioengineering12121297},
}
Figure