Structure-Guided MR-to-CT Synthesis with Spatial and Semantic Alignments for Attenuation Correction of Whole-Body PET/MR Imaging

Jiaxu Zheng, Zhenrong Shen, Lichi Zhang, Qun Chen

Abstract

Deep-learning-based MR-to-CT synthesis facilitates PET attenuation correction (AC) in whole-body PET/MR imaging by estimating tissue electron density. However, whole-body synthesis faces significant challenges, including spatial misalignment and complex intensity mapping across diverse tissues. This paper presents a novel framework comprising three modules: a Structure-Guided Synthesis (SGSyn) module for enhanced image quality via attention gates; a Spatial Alignment (SpatAlign) module for precise registration considering tissue volumes and respiratory motion; and a Semantic Alignment (SemAlign) module using contrastive learning for semantic authenticity. Extensive experiments demonstrate that the framework produces visually and semantically realistic CT images, significantly improving the accuracy of PET attenuation correction.

Introduction

Simultaneous PET/MR imaging is a powerful diagnostic tool, offering superior soft-tissue contrast and reduced radiation exposure compared to PET/CT. However, a major hurdle in PET/MR is Attenuation Correction (AC). In PET/CT, the CT scan provides a direct map of electron density used to correct the attenuation of gamma photons. MR signals, however, do not directly reflect tissue density, necessitating the synthesis of "pseudo-CT" images from MR data.

Synthesizing a CT from an MR for the whole body is exceptionally difficult due to three primary factors:

  1. Spatial Misalignment: MR and CT scans are rarely perfectly aligned due to differing acquisition times and patient positioning.
  2. Respiratory Motion: Organs like the liver "slide" against the chest wall, creating non-linear deformations that standard registration tools struggle to map.
  3. Complex Intensity Mapping: Soft tissues vary widely in MR intensity but appear nearly uniform in CT, often leading to "hallucinated" artifacts in synthetic images.

Methodology

The proposed framework addresses these challenges through a three-pronged deep-learning approach.

Structure-Guided Synthesis (SGSyn)

The core generative network is a modified 3D U-Net featuring an auxiliary "edge decoder." This decoder feeds into Structure-Guided Attention Gates, which instruct the network to prioritize anatomical boundaries. By focusing on these structural signals, the model effectively diminishes unnecessary soft-tissue contours that should not appear in a CT scan, resulting in cleaner, more accurate images.

Spatial Alignment (SpatAlign)

To ensure the ground-truth CT used for training matches the input MR, the authors developed a registration network with two innovative features:

  • Tissue-Aware Loss: This weights similarity loss inversely to tissue volume. It prevents large, uniform areas (like air or fat) from dominating the optimization, ensuring smaller, critical structures like ribs are accurately aligned.
  • Respiration-Aware Smoothness: Most algorithms assume the body deforms like a continuous object. SpatAlign identifies the "sliding" boundary between the liver and thoracic cavity and excludes it from typical smoothness constraints, allowing for more realistic, discontinuous motion modeling.

Semantic Alignment (SemAlign)

To ensure semantic authenticity, the framework utilizes a pre-trained segmentation model (MOOSE) to extract features from specific organs. It then employs Contrastive Learning (InfoNCE loss) to ensure that an organ in the synthetic CT (e.g., the liver) shares the same semantic features as the corresponding organ in the real CT. This prevents the model from confusing tissue types or hallucinating bone-like intensities inside soft organs.

Experimental Setup and Results

The study utilized a multi-center dataset of 350 subjects who underwent both whole-body PET/MR and PET/CT scans. 300 subjects were used for training, and 50 were reserved for testing. Images were resampled to 2mm isotropic voxels and normalized for consistency.

Image Quality

The proposed method was compared against state-of-the-art (SOTA) models, including CycleGAN and its variants. The framework achieved a PSNR of 21.71 and SSIM of 0.772, significantly outperforming the best CycleGAN variant (PSNR ~19.16, SSIM ~0.645). The improvement was most notable in bone structures, where synthetic CTs showed much clearer boundaries and more accurate Hounsfield Units (HU).

PET Attenuation Correction Utility

When applied to PET AC, the method resulted in the lowest Standardized Uptake Value (SUV) differences. In the spine and femur, SUV errors were nearly halved compared to traditional segmentation-based methods, validating the framework's clinical utility for precise diagnostic imaging.

Conclusion

This research introduces a robust solution for whole-body MR-to-CT synthesis by explicitly addressing the physical and semantic complexities of the human body. By combining structure-guided attention, respiration-aware registration, and semantic contrastive learning, the framework produces high-fidelity pseudo-CTs that significantly enhance PET attenuation correction. Future work will focus on reducing the dependency on paired training data through unsupervised or adaptive learning techniques.

Summary updated

Abstract

Deep-learning-based MR-to-CT synthesis can estimate the electron density of tissues, thereby facilitating PET attenuation correction in whole-body PET/MR imaging. However, whole-body MR-to-CT synthesis faces several challenges including the issue of spatial misalignment and the complexity of intensity mapping, primarily due to the variety of tissues and organs throughout the whole body. Here we propose a novel whole-body MR-to-CT synthesis framework, which consists of three novel modules to tackle these challenges: (1) Structure-Guided Synthesis module leverages structure-guided attention gates to enhance synthetic image quality by diminishing unnecessary contours of soft tissues; (2) Spatial Alignment module yields precise registration between paired MR and CT images by taking into account the impacts of tissue volumes and respiratory movements, thus providing well-aligned ground-truth CT images during training; (3) Semantic Alignment module utilizes contrastive learning to constrain organ-related semantic information, thereby ensuring the semantic authenticity of synthetic CT images.We conduct extensive experiments to demonstrate that the proposed whole-body MR-to-CT framework can produce visually plausible and semantically realistic CT images, and validate its utility in PET attenuation correction.
BibTeX
@article{zheng2025structure,
  author = {Jiaxu Zheng and Zhenrong Shen and Lichi Zhang and Qun Chen},
  title = {{Structure-Guided MR-to-CT Synthesis with Spatial and Semantic Alignments for Attenuation Correction of Whole-Body PET/MR Imaging}},
  journal = {Medical Image Analysis},
  year = {2025},
  doi = {10.1016/j.media.2025.103622},
}
Figure