End‐to‐End 3D CycleGAN Model for Amyloid PET Harmonization

Xiang-Kun Dong, Yalin Wang, Jay Shah, Valentina Ghisays, Ji Luo, Yinghua Chen, Wendy Lee, Baoxin Li, Kewei Chen, Eric M. Reiman, Teresa Wu, Yi SuAlzheimer 2024 · DOI

TL;DR

This paper introduces a novel 3D CycleGAN model to harmonize amyloid PET images across different tracers, specifically converting florbetapir (FBP) images to look like PiB images. This approach significantly improves image similarity and correlation, addressing a key challenge in multi-center Alzheimer's research by reducing the need for paired training data.

What Problem Does This Paper Address?

Alzheimer's disease research heavily relies on Positron Emission Tomography (PET) scans to detect amyloid burden in the brain. However, different PET tracers are used across various research centers, leading to variations in image appearance and making it difficult to combine data from different studies. This "tracer diversity" and the scarcity of paired data (where the same patient has scans from multiple tracers) pose a significant hurdle for large-scale collaborative research. The authors aim to solve this by creating a method to "harmonize" PET images, making them appear as if they were acquired with a standard tracer (in this case, PiB).

Methods

The core of this work is a 3D Cycle-Consistent Generative Adversarial Network (CycleGAN). This is a type of deep learning model designed for image-to-image translation, particularly when paired training data is scarce. Here's a breakdown of their approach:

  • 3D CycleGAN: Unlike 2D CycleGANs, this model operates on the entire 3D volume of PET scans, capturing more spatial context.
  • End-to-End Transformation: The model learns to directly transform an input image (FBP) into a target image (PiB) without requiring intermediate steps or manual feature extraction.
  • Adversarial Training: The CycleGAN consists of two main components: a generator (which creates the synthetic images) and a discriminator (which tries to distinguish between real and synthetic images). They train against each other, pushing the generator to produce increasingly realistic images.
  • Resnet Generators and Multilayer Discriminators: They employ established deep learning architectures (Resnet for generators and multilayer discriminators) to enhance the model's ability to learn complex transformations.
  • Patch-Based Learning: To manage computational resources and focus on local image details, the model learns from smaller 3D patches of the PET scans rather than the entire volume at once.
  • Data Augmentation: Techniques were used to artificially increase the size and diversity of the training dataset, helping to balance the samples and improve robustness.

Datasets

  • Training Data:
    • 51 florbetapir (FBP) images
    • 604 PiB images
    • These images are from the Australian Imaging, Biomarkers and Lifestyle Study of Ageing (AIBL).
  • Independent Test Dataset:
    • 46 paired images from www.gaain.org/centiloid-project. This dataset is crucial for evaluating the model's performance on unseen data and demonstrating its generalization capabilities.

Key Insights and Ideas

  • Unpaired Image Translation for Medical Imaging: The success of CycleGAN in harmonizing medical images like PET scans, especially without requiring perfectly paired data, is a significant advancement. This opens doors for leveraging existing, diverse datasets more effectively.
  • 3D Context Matters: Applying CycleGAN in 3D rather than 2D likely captures more comprehensive anatomical and pathological information, leading to more accurate and realistic image transformations.
  • Bridging the Gap Between Tracers: The model effectively learns the complex, non-linear relationships between different PET tracers, allowing for the conversion of one tracer's signal to another's. This is crucial for meta-analyses and multi-center studies.
  • Eliminating Paired Data Requirement: This is a major practical advantage. Collecting paired PET scans is expensive and time-consuming. By removing this bottleneck, the model democratizes the use of large, diverse PET datasets.

Key Results

  • Visual Similarity: The synthetic PiB images generated from FBP were visually much more similar to real PiB images than the original FBP images.
  • Improved Voxel-wise Correlation: The correlation between FBP and real PiB images was 0.942. After harmonization, the correlation between the synthetic PiB and real PiB images improved significantly to 0.958 (p < 0.0001).
  • Enhanced Agreement in Mean Cortical SUVR (mcSUVR): The agreement for mcSUVR, a key quantitative measure, improved from r = 0.909 (FBP vs. real PiB) to r = 0.954 (synthetic PiB vs. real PiB) in the independent test dataset (p < 0.001).
  • Quantitative Image Quality Metrics: In the independent test dataset, the Structural Similarity Index Measure (SSIM) between synthetic and real PiB was 0.762, and the Peak Signal-to-Noise Ratio (PSNR) was 25.370, indicating good quality of the synthesized images.

Limitations and Future Work

  • Limitations:
    • The paper focuses on converting FBP to PiB. While effective, it's a specific tracer pair.
  • The performance metrics (SSIM, PSNR) are good but not perfect, suggesting room for further improvement in image fidelity.
  • Future Work:
    • The authors propose this as a novel end-to-end image transformation model for 3D PET image synthesis. Future work could explore its application to other tracer pairs or even other PET tracers for different diseases.
  • Further validation on larger and more diverse independent datasets would strengthen the findings.
  • Investigating the impact of this harmonization on downstream clinical tasks (e.g., diagnosis, disease progression prediction) would be valuable.

Summary updated

Abstract

Abstract Background Amyloid PET (Positron Emission Tomography) is crucial in detecting amyloid burden within the brain. However, the diversity of amyloid tracers and the scarcity of paired data significantly challenge the collaboration between cross‐center studies. In this research, we introduce a novel patch‐based 3D end‐to‐end image transformation model. This model works as a harmonization strategy, transferring the amyloid PET images from one tracer type to another. Method 51 florbetapir (FBP) and 604 PiB images from the Australian Imaging, Biomarkers and Lifestyle Study of Ageing (AIBL) were processed using established pipelines to extract regional standard uptake value ratios (SUVRs), mean cortical SUVRs (mcSUVRs), and SUVR images. 3D Cycle‐Consistent Generative Adversarial Networks (CycleGAN) was used to learn the end‐to‐end 3D image transformation using adversarial training strategies in conjunction with Resnet generators and multilayer discriminators within different tracer domains. Data augmentation techniques were applied to process the FBP images to balance the training samples and patch‐based learning was used throughout the experiment. The trained CycleGAN model was then applied to an independent dataset with 46 paired images from www.gaain.org/centiloid‐project for performance evaluation. Correlation analyses were conducted voxel‐wise and on mcSUVR, comparing the FBP/synthetic PiB to the true PiB data. The Structural Similarity Index Measure (SSIM) and Peak Signal‐to‐Noise Ratio (PSNR) were also evaluated between the synthetic and real PiB SUVR images. Result The synthetic PiB SUVR images were visually more similar to real PiB SUVR images than FBP. Voxel‐wise correlation improved from 0.942 between FBP and real PiB to 0.958 between the virtual and real PiB SUVR image (p &lt; 0.0001). The agreement of mcSUVR improved from r = 0.909 to r = 0.954 (p&lt;0.001) in the independent test dataset. The SSIM and PSNR between synthetic and real PiB are 0.762 and 25.370 in the independent dataset. Conclusion We proposed a novel end‐to‐end image transformation model for 3D PET image synthesis. The model finds the nonlinear mapping between different tracers and eliminates the requirement for paired training images. The result was confirmed using an independent dataset to demonstrate its effectiveness.
BibTeX
@inproceedings{dong2024endtoend,
  author = {Xiang-Kun Dong and Yalin Wang and Jay Shah and Valentina Ghisays and Ji Luo and Yinghua Chen and Wendy Lee and Baoxin Li and Kewei Chen and Eric M. Reiman and Teresa Wu and Yi Su},
  title = {{End‐to‐End 3D CycleGAN Model for Amyloid PET Harmonization}},
  booktitle = {Alzheimer},
  year = {2024},
  doi = {10.1002/alz.095403},
}
Figure