Direct Energy-resolving CT Imaging via Energy-integrating CT images using a Unified Generative Adversarial Network

Lisha Yao, Li Sui, Ziquan Wei, Yaohong Deng, Manman Zhu, Zhaoying Bian, Jing Huang, Qingwen Lyu, Dong Zeng, Jianhua Ma2019 · DOI

TL;DR

This paper introduces a novel deep learning approach using a unified Generative Adversarial Network (uGAN) to generate multiple energy-resolved CT (ErCT) images directly from standard energy-integrating CT (EiCT) images. This work aims to make the rich spectral information offered by ErCT more accessible, overcoming the current limitations of expensive and complex ErCT hardware.

What Problem Does This Paper Address?

Energy-resolving computed tomography (ErCT) offers significant advantages over traditional energy-integrating CT (EiCT) by providing energy-dependent material information, leading to improved contrast-to-noise ratios and dose efficiency. However, ErCT systems rely on advanced, costly photon-counting detectors, making them largely unavailable for clinical use and research. This scarcity of ErCT data limits its widespread adoption and study. The paper addresses this by proposing a method to synthesize multiple ErCT images from readily available EiCT images, effectively democratizing access to ErCT's spectral capabilities.

Methods

The core of the proposed method is a unified Generative Adversarial Network (uGAN). This network is designed to perform image-to-image translation, specifically converting EiCT images into multiple ErCT images across different energy bins. The uGAN comprises two main components:

  • Generator (G): This module takes an EiCT image and a target energy label as input and generates a synthetic ErCT image (fErCT) at that specific energy. It's trained to fool the discriminator.
  • Discriminator (D): This module has a dual role. It first tries to distinguish between real ErCT (rErCT) images and the generated fErCT images. Second, it acts as an auxiliary classifier, determining the energy domain (i.e., the specific energy bin) of a given image.

The training objective for the uGAN includes three key loss functions:

  1. Adversarial Loss: Encourages the generator to produce fErCT images that are indistinguishable from real ErCT images.
  2. Domain Classification Loss: Ensures that the generator can accurately translate an EiCT image into an ErCT image corresponding to the specified target energy domain.
  3. Reconstruction Loss (Cycle Consistency): This loss is crucial for preserving the fidelity of the original EiCT image information. It enforces that if an image is translated from EiCT to ErCT and then back to EiCT, it should closely resemble the original EiCT image. This is implemented using an L1 norm.

Datasets

The study utilized clinical patient CT image data. The training dataset consisted of 1384 slices each for 70keV, 90keV, 110keV, 130keV, and 140kVp (EiCT) energy bins, collected from both chest and abdomen. Data augmentation techniques like rotation and flipping were applied to increase the training samples to 8304 slices per energy bin. An additional 130 slices at 140kVp were used for testing.

Key Insights and Ideas

  • Bridging the Data Gap: The most significant insight is the successful demonstration that high-quality ErCT images can be synthesized from standard EiCT data using deep learning. This bypasses the need for specialized, expensive hardware.
  • Unified GAN for Multi-Energy Output: Instead of generating ErCT images for a single energy, the uGAN is designed to concurrently train on multiple energy bins and produce multiple ErCT outputs simultaneously. This aligns better with the clinical requirement for spectral information across a range of energies.
  • Importance of Cycle Consistency: The inclusion of a cycle consistency loss is vital. It prevents the generator from simply "hallucinating" spectral information without regard for the original input data, ensuring that the generated ErCT images retain the structural integrity of the EiCT source.

Key Results

The uGAN model demonstrated promising results in generating ErCT images.

  • Qualitative: Visual inspection showed that the generated fErCT images were visually similar to the ground truth ErCT images across the tested energy bins (70, 90, 110, and 130 keV).
  • Quantitative: Feature Similarity Index (FSIM) values for the estimated fErCT images were consistently high, all above 0.99 (e.g., 0.9979 for 70keV, 0.9942 for 130keV). This indicates a strong agreement between the generated and ground truth images.
  • Difference Analysis: The difference images between fErCT and ground truth revealed small discrepancies, particularly in high-intensity areas. The authors attribute this to the significant attenuation coefficient differences between EiCT and ErCT, which the network struggles to fully capture.

Limitations and Future Work

  • High-Intensity Area Reconstruction: The paper acknowledges that the uGAN model struggles to reconstruct fine structural details in high-intensity areas. This is a notable limitation, as accurate material characterization in these regions is often critical.
  • Future Directions: The authors propose exploring additional techniques to enhance ErCT estimation performance, specifically mentioning the integration of "prior-texture induced networks" to potentially improve the reconstruction of details in challenging areas. Further research could also focus on optimizing the network architecture and training strategies to better handle the complex physics of X-ray attenuation across different energies.

Summary updated

Abstract

Energy-resolving computed tomography (ErCT) has the ability to acquire energy-dependent measurements simultaneously and quantitative material information with improved contrast-to-noise ratio. Meanwhile, ErCT imaging system is usually equipped with an advanced photon counting detector, which is expensive and technically complex. Therefore, clinical ErCT scanners are not yet commercially available, and they are in various stage of completion. This makes the researchers less accessible to the ErCT images. In this work, we investigate to produce ErCT images directly from existing energy-integrating CT (EiCT) images via deep neural network. Specifically, different from other networks that produce ErCT images at one specific energy, this model employs a unified generative adversarial network (uGAN) to concurrently train EiCT datasets and ErCT datasets with different energies and then performs image-to-image translation from existing EiCT images to multiple ErCT image outputs at various energy bins. In this study, the present uGAN generates ErCT images at 70keV, 90keV, 110keV, and 130keV simultaneously from EiCT images at140kVp. We evaluate the present uGAN model on a set of over 1380 CT image slices and show that the present uGAN model can produce promising ErCT estimation results compared with the ground truth qualitatively and quantitatively.
BibTeX
@misc{yao2019direct,
  author = {Lisha Yao and Li Sui and Ziquan Wei and Yaohong Deng and Manman Zhu and Zhaoying Bian and Jing Huang and Qingwen Lyu and Dong Zeng and Jianhua Ma},
  title = {{Direct Energy-resolving CT Imaging via Energy-integrating CT images using a Unified Generative Adversarial Network}},
  year = {2019},
  doi = {10.1109/nss/mic42101.2019.9060060},
}
Figure