Q-space Conditioned Translation Networks for Directional Synthesis of Diffusion Weighted Images from Multi-modal Structural MRI

Mengwei Ren, Heejong Kim, Neel Dey, Guido Gerig

TL;DR

Researchers have developed a generative adversarial network (GAN) that synthesizes high-quality Diffusion Weighted Images (DWIs) from standard structural MRI scans (T1, T2, and B0). By conditioning the network on continuous $q$-space information, the model can generate DWIs for any arbitrary gradient direction or b-value, effectively "filling in the blanks" for sparse or corrupted diffusion datasets.

What Problem Does This Paper Address?

Diffusion MRI (dMRI) is a powerful tool for looking at the microscopic structure of the brain by measuring how water molecules move. While traditional measures like Fractional Anisotropy (FA) are common, newer models like NODDI and DKI provide much deeper insights into tissue configuration. The catch? These advanced models require a massive amount of data—densely sampled "$q$-space" (many different directions and strengths of magnetic gradients)—which leads to long scan times.

Long scans are prone to patient motion and technical artifacts. When data is corrupted, researchers often have to throw out specific gradient directions, leaving "holes" in the data. While deep learning has been used to predict microstructural maps from undersampled data, most existing models assume a static sampling pattern. If your real-world clinical scan doesn't match the exact sampling pattern the AI was trained on, the results degrade. This paper addresses the need for a flexible synthesis tool that works with any arbitrary sampling scheme.

Methods

The authors propose a $q$-space Conditioned Translation Network based on a GAN framework. The architecture consists of two main components:

  1. $Q$-space Conditioned Generator: Instead of just taking T1/T2 images as input, the generator is "told" which specific $q$-space coordinate (b-vector and b-value) it needs to produce. It uses Feature-wise Linear Modulation (FiLM) to inject this coordinate information into the network's internal layers. This allows the network to linearly modulate its representations, making it capable of synthesizing images for a continuous range of directions rather than a fixed set.
  2. Conditional U-Net Discriminator: Unlike standard GAN discriminators that look at small patches, this model uses a U-Net structure. The bottleneck assesses global realism (does this look like a brain?), while the output feature map assesses pixel-level realism (are the details sharp?).

The team also incorporated physics-informed data augmentation. Since diffusion signals from opposite directions (antipodal) are ideally identical, they trained the model to recognize this symmetry. They also ensured that if the b-value is zero, the model correctly outputs the baseline (B0) image.

Datasets

The study utilized the HCP 500 release (Human Connectome Project), specifically focusing on 19 unrelated subjects.

  • Training: 9 subjects.
  • Validation: 1 subject.
  • Testing: 9 held-out subjects.
  • Specs: The data included DWIs at b-values of 1000, 2000, and 3000 $s/mm^2$ across roughly 270 gradient directions, with a voxel size of 1.25mm³.

Key Insights and Ideas

  • Synthesis First, Estimation Second: A major insight is that synthesizing the missing DWIs before calculating microstructural indices (like NDI or MK) is more effective than trying to regress those indices directly from structural MRI. This preserves the "downstream utility" of the data, allowing researchers to use any diffusion model they want after the synthesis.
  • The Power of Multi-modal Inputs: The paper demonstrates a clear "staircase" of improvement: using B0 alone is okay, adding T2 helps, and adding T1 provides the best structural fidelity.
  • Handling "Unpredictable" Corruption: Because the model is conditioned on continuous $q$-space coordinates, it can handle the "messy" reality of clinical pipelines where gradients are reoriented or excluded due to motion and eddy currents.

Key Results

The proposed model (Model D: B0+T1+T2 + GAN) outperformed several baselines in both image quality and downstream accuracy:

  • Image Fidelity: It achieved a Peak Signal-to-Noise Ratio (PSNR) of 29.77 and a Structural Similarity Index (SSIM) of 0.944, significantly higher than non-adversarial or single-modality models.
  • Microstructure Accuracy: When tested on heavily downsampled data (e.g., only 13 or 26 DWIs), the synthesized images allowed for much more accurate estimation of NODDI and DKI parameters compared to standard model-fitting or other deep learning approaches (q-DL).
  • Qualitative Sharpness: Visually, the GAN-based approach recovered high-frequency details (like the external capsule) that were blurred out by simpler regression models.

Limitations and Future Work

The authors acknowledge a few key hurdles for real-world clinical adoption:

  • Healthy Cohort Bias: The model was trained and tested on a healthy population. It is currently unknown how well it would synthesize DWIs in the presence of pathology, such as brain lesions or tumors, which might have different diffusion properties.
  • Registration and Resolution: The framework assumes T1, T2, and B0 images are perfectly aligned. In practice, inter-modality misregistration and differences in resolution could impede performance.
  • Validation on Diverse Data: Future work will need to validate the model on larger, more neuroanatomically diverse cohorts to ensure the "learned" mappings generalize across different scanners and populations.

Summary updated

Abstract

Current deep learning approaches for diffusion MRI modeling circumvent the need for densely-sampled diffusion-weighted images (DWIs) by directly predicting microstructural indices from sparsely-sampled DWIs. However, they implicitly make unrealistic assumptions of static q-space sampling during training and reconstruction. Further, such approaches can restrict downstream usage of variably sampled DWIs for usages including the estimation of microstructural indices or tractography. We propose a generative adversarial translation framework for high-quality DWI synthesis with arbitrary q-space sampling given commonly acquired structural images (e.g., B0, T1, T2). Our translation network linearly modulates its internal representations conditioned on continuous q-space information, thus removing the need for fixed sampling schemes. Moreover, this approach enables downstream estimation of high-quality microstructural maps from arbitrarily subsampled DWIs, which may be particularly important in cases with sparsely sampled DWIs. Across several recent methodologies, the proposed approach yields improved DWI synthesis accuracy and fidelity with enhanced downstream utility as quantified by the accuracy of scalar microstructure indices estimated from the synthesized images. Code is available at https://github.com/mengweiren/q-space-conditioned-dwi-synthesis .
BibTeX
@inproceedings{ren2021q,
  author = {Mengwei Ren and Heejong Kim and Neel Dey and Guido Gerig},
  title = {{Q-space Conditioned Translation Networks for Directional Synthesis of Diffusion Weighted Images from Multi-modal Structural MRI}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87234-2\_50},
}

Loading related content...

Figure