Structure-guided MR-to-CT synthesis with spatial and semantic alignments for attenuation correction of whole-body PET/MR imaging

Jiaxu Zheng, Zhenrong Shen, Lichi Zhang, Qun Chen

TL;DR

Researchers have developed a novel deep-learning framework that synthesizes high-fidelity whole-body CT scans from MR images. By combining structure-guided attention, respiration-aware spatial alignment, and semantic contrastive learning, the system produces "pseudo-CTs" accurate enough to perform precise attenuation correction for PET/MR imaging, significantly outperforming existing GAN-based methods.

What Problem Does This Paper Address?

Simultaneous PET/MR imaging is a powerful diagnostic tool, but it has a major hurdle: Attenuation Correction (AC). In PET/CT, the CT scan provides a map of electron density used to correct for the "dimming" effect of tissues on gamma photons. MR scans, however, do not directly reflect tissue density.

Synthesizing a CT from an MR (MR-to-CT synthesis) is the preferred solution, but doing this for the whole body is incredibly difficult. Unlike the head or pelvis, the whole body involves:

  1. Spatial Misalignment: MR and CT scans are rarely perfectly aligned due to different acquisition times and patient positioning.
  2. Respiratory Motion: Organs like the liver "slide" against the chest wall, creating non-linear deformations that standard registration tools struggle to map.
  3. Complex Intensity Mapping: Soft tissues vary wildly in MR intensity but appear nearly uniform in CT, leading to "hallucinated" artifacts in synthetic images.

Methods

The authors propose a three-pronged framework to tackle these challenges:

  • Structure-Guided Synthesis (SGSyn): This is the core generative network (a modified 3D U-Net). It features an auxiliary "edge decoder" that feeds into Structure-Guided Attention Gates. These gates tell the network to prioritize anatomical boundaries, effectively "cleaning up" unnecessary soft-tissue contours that shouldn't appear in a CT.
  • Spatial Alignment (SpatAlign): To ensure the ground-truth CT used for training actually matches the input MR, the authors use a registration network with two clever tweaks:
  • Tissue-Aware Loss: It weights the similarity loss inversely to tissue volume, preventing large areas (like air or fat) from drowning out smaller, critical structures like ribs.
  • Respiration-Aware Smoothness: It identifies the "sliding" boundary between the liver and thoracic cavity and excludes it from the typical smoothness constraint, allowing for more realistic, discontinuous motion.
  • Semantic Alignment (SemAlign): Using a pre-trained segmentation model (MOOSE), the framework extracts features from specific organs. It then uses Contrastive Learning (InfoNCE loss) to ensure that a "liver" in the synthetic CT has the same semantic features as a "liver" in the real CT, preventing the model from confusing different tissue types.

Datasets

The study utilized a multi-center dataset of 350 subjects who underwent both whole-body PET/MR (UIH uPMR790) and PET/CT (UIH uMI780) scans.

  • Training: 300 subjects.
  • Testing: 50 subjects.
  • Preprocessing: Images were resampled to 2mm isotropic voxels and normalized. The regions above the shoulders were cropped to avoid artifacts from varying arm positions.

Key Insights and Ideas

  • The "Sliding" Problem: Most registration algorithms assume the body deforms like a continuous piece of rubber. By explicitly masking out the thoracic-abdominal interface, the authors allowed the model to learn the "slippage" caused by breathing, leading to much better alignment of the ribs and liver.
  • Semantic Authenticity: Simply matching pixels isn't enough. By forcing the network to learn organ-specific features via contrastive learning, the model avoids "hallucinating" bone-like intensities inside soft organs.
  • Attention to Edges: Using an edge-map as a guiding signal helps the generator distinguish between actual anatomical boundaries and the complex intensity variations inherent in MR scans.

Key Results

The proposed method was compared against state-of-the-art (SOTA) models like CycleGAN and its variants (gc-CycleGAN, sc-CycleGAN).

  • Image Quality: The method achieved a PSNR of 21.71 and SSIM of 0.772, significantly higher than the best-performing CycleGAN variant (PSNR ~19.16, SSIM ~0.645).
  • Organ-Specific Accuracy: The improvement was most notable in bone structures (Ribs and Femur), where the synthetic CTs showed much clearer boundaries and more accurate Hounsfield Units (HU).
  • PET Attenuation Correction: When used for PET AC, the method resulted in the lowest Standardized Uptake Value (SUV) differences. In the spine and femur, the SUV errors were nearly halved compared to traditional segmentation-based methods.

Limitations and Future Work

  • Reliance on Paired Data: While SpatAlign handles misalignment, the framework still requires paired MR and CT data for the initial training phase, which can be difficult to collect at scale.
  • Segmentation Dependency: The SemAlign module relies on the accuracy of the pre-trained MOOSE segmentation model. If the segmentation fails on a specific organ, the semantic constraint might be weakened.
  • Future Directions: The authors suggest moving toward unsupervised or adaptive learning to reduce the dependency on paired data while maintaining the high structural and semantic fidelity demonstrated here.

Summary updated

Abstract

Deep-learning-based MR-to-CT synthesis can estimate the electron density of tissues, thereby facilitating PET attenuation correction in whole-body PET/MR imaging. However, whole-body MR-to-CT synthesis faces several challenges including the issue of spatial misalignment and the complexity of intensity mapping, primarily due to the variety of tissues and organs throughout the whole body. Here we propose a novel whole-body MR-to-CT synthesis framework, which consists of three novel modules to tackle these challenges: (1) Structure-Guided Synthesis module leverages structure-guided attention gates to enhance synthetic image quality by diminishing unnecessary contours of soft tissues; (2) Spatial Alignment module yields precise registration between paired MR and CT images by taking into account the impacts of tissue volumes and respiratory movements, thus providing well-aligned ground-truth CT images during training; (3) Semantic Alignment module utilizes contrastive learning to constrain organ-related semantic information, thereby ensuring the semantic authenticity of synthetic CT images.We conduct extensive experiments to demonstrate that the proposed whole-body MR-to-CT framework can produce visually plausible and semantically realistic CT images, and validate its utility in PET attenuation correction.
BibTeX
@article{zheng2025structure,
  author = {Jiaxu Zheng and Zhenrong Shen and Lichi Zhang and Qun Chen},
  title = {{Structure-guided MR-to-CT synthesis with spatial and semantic alignments for attenuation correction of whole-body PET/MR imaging}},
  journal = {Medical Image Analysis},
  year = {2025},
  doi = {10.1016/j.media.2025.103622},
}

Loading related content...

Figure