Neural Contrast Enhancement of CT Image

Minkyo Seo, Dongkeun Kim, Kyungmoon Lee, Seunghoon Hong, Jae Seok Bae, Jung Hoon Kim, Suha KwakWACV 2021 · OpenReview

Abstract

Contrast materials are often injected into the body to contrast specific tissues in Computed Tomography (CT) images. Contrast-enhanced CT (CECT) images obtained in this way are more useful than non-enhanced CT (NECT) images for medical diagnosis, but are not available for everyone due to side effects of the contrast materials. Motivated by this, we develop a neural network that takes NECT images and generates their CECT counterparts. Learning such a network is extremely challenging since NECT and CECT images for training are not aligned even at the same location of the same patient due to movements of internal organs. We propose a two-stage framework to address this issue. The first stage trains an auxiliary network that removes the effect of contrast enhancement in CECT images to synthesize their NECT counterparts well-aligned with them. In the second stage, the target model is trained to predict the real CECT images given a synthetic NECT image as input. Experimental results and analysis by physicians on abdomen CT images suggest that our method outperforms existing models for neural image synthesis.

Introduction

Contrast-enhanced CT (CECT) is the gold standard for diagnosing many conditions because injected contrast agents highlight blood vessels and organs. However, these agents are not suitable for all patients: they can cause allergic reactions, pose risks for those with kidney disease, and increase radiation exposure and cost.

While synthesizing CECT from standard non-enhanced CT (NECT) images using deep learning is a promising solution, it faces two major hurdles. First, misalignment: because NECT and CECT scans are taken at different times, internal organs move due to breathing or digestion, preventing simple pixel-to-pixel comparison. Second, patient variability: contrast enhancement patterns vary wildly depending on a patient’s specific medical condition.

Misalignment between NECT and CECT scans due to organ movement.

Methodology: A Two-Stage Framework

To bypass the alignment issue, we propose a two-stage training strategy that leverages an auxiliary task to create perfectly aligned training pairs.

Stage 1: The Inverse Task

Instead of directly mapping NECT to CECT, we first train an auxiliary generator ($G_{C \to N}$) to remove contrast from real CECT images. This "de-enhancement" task is more stable because NECT images exhibit a more consistent, monotonic appearance across different patients. The output of this stage is a synthetic NECT image that is perfectly aligned with the real CECT image from which it was derived.

Stage 2: The Target Task

With aligned pairs of synthetic NECT and real CECT images, we train the main generator ($G_{N \to C}$) using reconstruction losses (L1 and Perceptual loss). Because the images are perfectly aligned, the model can learn exactly which pixels should be enhanced without being confused by organ shifts.

The proposed two-stage framework for generating CECT images from NECT.

Architecture and Supervision

Both generators utilize Spatially-Adaptive Denormalization (SPADE) blocks, which excel at preserving semantic layout while transforming textures. To provide stronger adversarial supervision, we employ an AM-GAN discriminator that classifies images into three categories: Real NECT, Real CECT, or Synthetic NECT. This combination ensures that the underlying anatomy isn't warped during the synthesis process, a common failure point in standard image-to-image translation models.

Experimental Results

The framework was evaluated on an abdomen CT dataset containing 23,923 image pairs and an IXI Brain MRI dataset with simulated distortions to measure spatial error handling.

  • Quantitative Performance: On the IXI Brain MRI benchmark, our method consistently outperformed CycleGAN and single-stage baselines in terms of Mean Squared Error (MSE) and Multi-Scale Structural Similarity (MS-SSIM).
  • Qualitative Physician Review: Radiologists evaluated the synthetic images, giving our method significantly higher scores for "Image Quality" and lower scores for "Degree of Artifact" compared to CycleGAN. Physicians reported that the synthetic CECTs were more useful for identifying blood vessels and organ boundaries than the original NECTs, without the "hallucinated" textures often seen in AI-generated images.

Conclusion

Our two-stage framework effectively addresses the misalignment challenge in medical image synthesis. By using an auxiliary "inverse" task to generate aligned training data, we produce high-quality synthetic CECT images that offer clinical utility without the risks associated with contrast agents. Future work will focus on improving the synthesis of small, highly variable lesions and exploring joint optimization of registration and synthesis.

Summary updated

Abstract

Contrast materials are often injected into body to contrast specific tissues in Computed Tomography (CT) images. Contrast Enhanced CT (CECT) images obtained in this way are more useful than Non-Enhanced CT (NECT) images for medical diagnosis, but not available for everyone due to side effects of the contrast materials. Motivated by this, we develop a neural network that takes NECT images and generates their CECT counterparts. Learning such a network is extremely challenging since NECT and CECT images for training are not aligned even at the same location of the same patient due to movements of internal organs. We propose a two-stage framework to address this issue. The first stage trains an auxiliary network that removes the effect of contrast enhancement in CECT images to synthesize their NECT counterparts well-aligned with them. In the second stage, the target model is trained to predict the real CECT images given a synthetic NECT image as input. Experimental results and analysis by physicians on abdomen CT images suggest that our method outperforms existing models for neural image synthesis.
BibTeX
@inproceedings{seo2021neural,
  author = {Minkyo Seo and Dongkeun Kim and Kyungmoon Lee and Seunghoon Hong and Jae Seok Bae and Jung Hoon Kim and Suha Kwak},
  title = {{Neural Contrast Enhancement of CT Image}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=l5ibjccSay},
}
Figure