TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Xueqi Guo, Luyao Shi, Xiongchao Chen, Qiong Liu, Bo Zhou, Huidong Xie, Yi-Hwa Liu, Richard Palyo, Edward J. Miller, Albert J. Sinusas, Lawrence H. Staib, Bruce Spottiswoode, Chi Liu, Nicha C. Dvornek

Abstract

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 ($^{82}$Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical $^{82}$Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames.

Introduction

Dynamic cardiac Positron Emission Tomography (PET) using rubidium-82 ($^{82}$Rb) is a gold standard for measuring myocardial blood flow (MBF). However, the acquisition process, which lasts several minutes, is susceptible to inter-frame motion from patient movement or respiration. Correcting this motion is challenging because the tracer distribution changes rapidly over time. In early frames, the tracer is concentrated in the blood pools, while in late frames, it moves into the myocardium. This "blood-pool to tissue" transition makes standard intensity-based registration algorithms fail, leading to inaccurate MBF quantification.

Methods

The proposed TAI-GAN framework addresses this by converting all early frames into synthetic versions that mimic the tracer distribution of the final reference frame. This transforms a complex multi-modal registration problem into a simpler intra-modal one.

The TAI-GAN architecture featuring a generator with FiLM layers for temporal integration and a discriminator for adversarial training.

Key technical innovations include:

  • All-to-One Mapping: A single model trained to map any early frame to the distribution of the last frame.
  • Temporal Information (FiLM): The model uses an LSTM network to process Time-Activity Curves (TACs)—data showing how tracer concentration changes over time. This info is injected into the generator via Feature-wise Linear Modulation (FiLM) layers, helping the network understand the tracer's lifecycle.
  • Anatomical Guidance: The network receives rough segmentation masks of the heart chambers and muscle. To ensure robustness against pre-existing motion, these masks are subjected to "local shifts" during training.

Results

The TAI-GAN was evaluated on a clinical dataset of 104 $^{82}$Rb PET scans from Yale New Haven Hospital. The model demonstrated superior performance in synthesizing high-quality late-phase frames from early-phase data.

Visual comparison of early frames converted to late-frame distributions using various GAN architectures. TAI-GAN shows the highest fidelity to the real reference frame.
  • Image Quality: TAI-GAN achieved a Structural Similarity Index (SSIM) of 0.733, outperforming baseline GANs (0.708).
  • Motion Correction Accuracy: In simulated motion benchmarks, TAI-GAN reduced mean absolute motion prediction error from 5.30 mm to 4.70 mm.
  • Clinical Impact on MBF: The most vital result was in MBF quantification. Standard motion correction (BIS) without conversion had a 14.45% error; with TAI-GAN conversion, that error dropped to just 2.21%.
  • Efficiency: The all-to-one approach is roughly 8 times faster to train than approaches requiring separate models for every time point.

Conclusion

TAI-GAN effectively bridges the gap between early and late PET frames by incorporating tracer kinetics and anatomical priors. While currently limited to post-reconstruction images and requiring rough manual segmentations, it significantly enhances the reliability of MBF quantification. Future work will focus on automating the segmentation process and addressing attenuation mismatches.

Summary updated

Abstract

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82-Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82-Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames.
BibTeX
@article{guo2024tai,
  author = {Xueqi Guo and Luyao Shi and Xiongchao Chen and Qiong Liu and Bo Zhou and Huidong Xie and Yi-Hwa Liu and Richard Palyo and Edward J. Miller and Albert J. Sinusas and Lawrence H. Staib and Bruce Spottiswoode and Chi Liu and Nicha C. Dvornek},
  title = {{TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction}},
  journal = {Medical Image Analysis},
  year = {2024},
  url = {https://openreview.net/forum?id=zAaZSKYZ9S},
}
Figure