2S-CEDiff: A Two-Stage Diffusion Framework for Generating High-Fidelity Contrast-Enhanced CT Images from Non-Contrast Scans

Yibang Wu, Tzung-Dau Wang, Shang-Hong Lai

TL;DR

This paper introduces 2S-CEDiff, a novel two-stage framework that generates high-fidelity contrast-enhanced CT (CECT) images from non-contrast CT (NCCT) scans. By combining a 2.5D diffusion model with a lightweight 3D refinement network, 2S-CEDiff achieves superior anatomical accuracy and volumetric consistency, crucial for medical diagnostics, without the prohibitive computational cost of full 3D diffusion models.

What Problem Does This Paper Address?

Contrast-enhanced Computed Tomography (CECT) is vital for medical imaging, particularly in cardiovascular assessment, as it highlights vascular structures and pathological regions. However, the use of intravenous contrast agents carries potential health risks for certain patient populations. This has driven research into image-to-image translation techniques that can synthesize CECT images from readily available non-contrast CT (NCCT) scans. Existing methods often struggle to balance high image fidelity, consistent volumetric structure, and computational efficiency, especially for complex anatomical regions like the heart's myocardium. Previous approaches using 2D models often suffer from poor inter-slice consistency, leading to artifacts, while full 3D models are computationally expensive and memory-intensive, limiting their clinical applicability.

Methods

The proposed 2S-CEDiff framework employs a two-stage approach:

  • Stage 1: 2.5D Diffusion-based Slice Generation: This stage utilizes a conditional diffusion model to generate anatomically accurate, contrast-enhanced slices from NCCT inputs. It incorporates a local 3D context by using a 5-slice window of NCCT images. Crucially, it integrates:
  • Anatomical Mask Guidance: Pre-computed segmentation masks from a tool called TotalSegmentator are used to guide the diffusion process, emphasizing clinically relevant regions.
  • Cross-Attention Guidance: Positional embeddings (slice index) and brightness differences between adjacent slices are used as weighting maps in cross-attention layers. This helps the model adaptively focus on areas requiring enhancement, accounting for anatomical variability and subtle contrast changes along the superior-inferior axis.
  • ROI-Weighted Loss: A modified loss function is used to prioritize accurate reconstruction within the anatomically guided regions.
  • Stage 2: Lightweight 3D Refinement: To address the inter-slice inconsistencies that can arise from the 2.5D approach, a lightweight 3D U-Net is employed. This network takes the generated slices from Stage 1 and their corresponding anatomical masks as input. It refines the volume using 3D convolutional blocks but preserves the slice (depth) dimension to maintain inter-slice continuity. This stage is applied using a sliding window strategy. The loss function for this stage combines:
  • Pixel-wise L₁ Loss: Standard mean absolute error between predicted and ground truth slices.
  • Z-axis SSIM Consistency Loss: Penalizes structural dissimilarities between adjacent slices in the refined output.
  • Volumetric SSIM Loss: Quantifies the overall 3D structural similarity of the refined volume segment.

This hybrid approach aims to leverage the generative power of diffusion models while enforcing volumetric consistency with significantly reduced computational overhead compared to full 3D diffusion.

Datasets

The authors trained and evaluated their method on:

  • HeartCT Dataset: A private, de-identified 3D cardiac CT dataset comprising 150 paired volumes of native (NCCT) and arterial-phase contrast-enhanced (CECT) scans. The data has a resolution of 256x256x256 with 0.5mm isotropic voxel spacing. The dataset was split into 110 training, 20 validation, and 20 testing pairs. This dataset is not publicly available.
  • Coltea-Lung-CT-100W Dataset: A public dataset containing 100 thoracic CT scans from female patients acquired in three phases (NCCT, early portal venous, and late arterial CECT). For this study, NCCT-CECT pairs were used, cropped around the heart to 256x256 patches.
  • BraTS2021 Dataset: A benchmark dataset for multi-modal brain MRI scans. The authors framed a T1-to-T2 translation task, using T1 as input and T2 as target, to test generalizability across modalities.

Key Insights and Ideas

  • Hybrid 2.5D + 3D Approach: The core innovation is the combination of a 2.5D diffusion model for initial synthesis with a lightweight 3D refinement network. This elegantly balances the strengths of diffusion models (fidelity) with the need for volumetric coherence, while avoiding the computational burden of full 3D diffusion.
  • Anatomical and Brightness Guidance: Integrating segmentation masks and slice-wise brightness differences as conditioning signals significantly improves the model's ability to generate anatomically accurate and contextually appropriate contrast enhancement, especially in challenging regions.
  • Focus on Myocardium: The paper highlights the particular difficulty in accurately enhancing the myocardium due to its thin walls and proximity to other structures. Their method shows significant improvements in segmenting this critical region, underscoring its clinical relevance.
  • Efficient Clinical Translation: By achieving state-of-the-art results with significantly reduced computational cost and memory requirements compared to full 3D models, 2S-CEDiff is positioned for practical clinical deployment.

Key Results

  • Image Quality: 2S-CEDiff achieved state-of-the-art performance on the HeartCT dataset, with PSNR of 29.87 and SSIM of 0.891. It also showed strong results on the Coltea and BraTS2021 datasets.
  • Segmentation Accuracy: On the HeartCT dataset, 2S-CEDiff achieved the highest Dice scores across seven cardiac structures, with a notable improvement in myocardium segmentation, increasing the Dice score to 0.9447 and yielding a +0.0798 gain compared to native NCCT input.
  • Inter-slice Consistency: The Stage 2 refinement module significantly improved inter-slice consistency, reducing the Average Inter-slice Intensity Difference (ISID) to 1.074, compared to 3.029 for 2D DDPM, resulting in smoother and more coherent volumes.
  • Computational Efficiency: The two-stage framework is computationally efficient. Training took 15 hours, and inference per volume was 64 seconds on a single NVIDIA RTX 4090 GPU, which is substantially faster than full 3D DDPMs (400s inference per volume) and more practical than pure 3D diffusion models.

Limitations and Future Work

  • Dataset Dependency: While the BraTS2021 results show generalizability, the primary evaluation was on cardiac CT datasets. Further validation on diverse CT datasets and modalities would be beneficial.
  • Reliance on Segmentation Masks: The method relies on pre-computed segmentation masks from TotalSegmentator. While this is a practical approach, the performance is inherently tied to the accuracy of these masks.
  • Future Work: The authors suggest exploring more advanced 3D refinement modules or integrating the 3D refinement more deeply into the diffusion process itself for even greater volumetric coherence. Further investigation into optimizing the framework for real-time inference in clinical settings is also a potential direction.

Summary updated

BibTeX
@inproceedings{wu2026s,
  author = {Yibang Wu and Tzung-Dau Wang and Shang-Hong Lai},
  title = {{2S-CEDiff: A Two-Stage Diffusion Framework for Generating High-Fidelity Contrast-Enhanced CT Images from Non-Contrast Scans}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2026},
}
Figure