Multi-Contrast MRI Segmentation Trained on Synthetic Images

Ismail Irmakci, Zeki Emre Unel, Nazli Ikizler-Cinbis, Ulas Bagci

Abstract

In clinical radiology, multi-contrast Magnetic Resonance Imaging (MRI) provides a comprehensive view of tissue properties, yet incomplete datasets frequently hinder automated analysis. This study investigates the feasibility of using Generative Adversarial Networks (GANs) to synthesize missing MRI contrasts for training deep learning segmentation models. By comparing models trained on real versus synthetic data for thigh tissue segmentation, the researchers demonstrate that synthetic images can effectively replace real data with minimal loss in accuracy, offering a potential solution to the data scarcity bottleneck in medical imaging.

Introduction

Multi-contrast MRI is a powerful diagnostic tool that allows clinicians to visualize diverse tissue properties—such as fat, water, and bone—by varying acquisition parameters. However, clinical workflows are often imperfect; patient time constraints, technical errors, or cost limitations frequently result in missing or incomplete contrast sets. This inconsistency poses a significant challenge for automated segmentation algorithms, which typically require complete multi-contrast inputs to achieve high precision.

Automated segmentation is essential for quantifying health metrics, such as muscle mass in sarcopenia studies or adipose tissue distribution in obesity research. To address the dual challenges of missing data and the high cost of labeled medical datasets, Irmakci et al. proposed a pipeline that uses synthetic data generation to augment or replace real training sets.

Methodology

The researchers employed a two-stage deep learning pipeline to test the utility of synthetic images:

  1. Image Synthesis: The study utilized pix2pix, a conditional Generative Adversarial Network (cGAN) framework, to perform image-to-image translation. The GAN was trained to learn the mapping between different MRI contrasts, effectively "translating" one contrast (e.g., fat-suppressed) into another (e.g., water-suppressed).
  2. Tissue Segmentation: A standard U-Net architecture was used for the segmentation task. The U-Net was tasked with identifying four primary thigh tissues: muscle, fat, bone, and bone marrow.

To evaluate the effectiveness of synthetic data, the authors conducted an ablation study comparing three training regimes: models trained exclusively on real images, models trained exclusively on GAN-generated synthetic images, and models trained on a mixture of both.

Experimental Setup and Dataset

The study utilized data from the Baltimore Longitudinal Study of Aging (BLSA), comprising 150 volumetric MRI scans from 50 subjects. Scans were performed on a 3T Philips Achieva MRI scanner, focusing on three specific T1-weighted variations:

  • MRI1: Fat-suppressed images.
  • MRI2: Water and fat images.
  • MRI3: Water-suppressed images.

Results and Discussion

The quantitative results revealed that synthetic training is a remarkably viable alternative to using real data. The performance gap between models trained on real images and those trained on synthetic images was consistently narrow, often less than 1–2% in terms of the Dice Similarity Coefficient (DSC).

Segmentation Performance

  • Muscle Segmentation: The synthetic-trained model achieved a DSC of 93.91%, compared to 94.68% for the real-trained model.
  • Bone Marrow Segmentation: The synthetic-trained model reached 95.33% DSC, while the real-trained model achieved 96.82%.

Synthesis Quality

The GAN-generated images showed high fidelity to the original contrasts, achieving a Peak Signal-to-Noise Ratio (PSNR) of 28.53 and a Structural Similarity Index (SSIM) of 0.89. Qualitatively, the GAN successfully preserved intricate anatomical details in soft tissues, which are critical for accurate boundary detection in segmentation tasks.

Key Insights

The study highlighted that certain MRI contrasts share underlying features that allow for highly effective one-to-one mapping (specifically MRI1 to MRI3). Furthermore, the researchers found that multi-contrast synergy persists even when some inputs are synthetic. Providing a model with multiple inputs (e.g., one real and two synthetic) generally outperformed single-contrast segmentation, suggesting that the "hallucinated" information follows learned physical patterns that aid the model in resolving complex tissue boundaries.

Conclusion and Future Work

This research demonstrates that deep learning models do not strictly require real multi-contrast datasets to achieve clinical-grade segmentation accuracy. Synthetic data generated via GANs can capture the necessary anatomical and physical relationships to serve as a robust training proxy.

While these results are promising for musculoskeletal radiology, future research is needed to determine if this approach translates to more heterogeneous anatomical regions, such as the brain or abdomen, or to more complex pathologies. Nonetheless, this methodology provides a clear path forward for large-scale training of deep networks in scenarios where complete multi-contrast data is unavailable.

Summary updated

Abstract

In our comprehensive experiments and evaluations, we show that it is possible to generate multiple contrast (even all synthetically) and use synthetically generated images to train an image segmentation engine. We showed promising segmentation results tested on real multi-contrast MRI scans when delineating muscle, fat, bone and bone marrow, all trained on synthetic images. Based on synthetic image training, our segmentation results were as high as 93.91%, 94.11%, 91.63%, 95.33%, for muscle, fat, bone, and bone marrow delineation, respectively. Results were not significantly different from the ones obtained when real images were used for segmentation training: 94.68%, 94.67%, 95.91%, and 96.82%, respectively. Clinical relevance— Synthetically generated images could potentially be used in large-scale training of deep networks for segmentation purpose. Small data set problem of many clinical imaging problems can potentially be addressed with the proposed algorithm.
BibTeX
@inproceedings{irmakci2022multi,
  author = {Ismail Irmakci and Zeki Emre Unel and Nazli Ikizler-Cinbis and Ulas Bagci},
  title = {{Multi-Contrast MRI Segmentation Trained on Synthetic Images}},
  booktitle = {EMBC},
  year = {2022},
  url = {https://openreview.net/forum?id=4Zd6pThrnT},
}
Figure