MR Slice Profile Estimation by Learning to Match Internal Patch Distributions

Shuo Han, Samuel Remedios, Aaron Carass, Michael Schär, Jerry L. Prince

TL;DR

This paper introduces a novel method to estimate the "slice profile" of MRI scans, which describes how the image is blurred in the through-plane direction. By learning to match internal image patch distributions across different orientations, their GAN-based approach can infer this profile directly from the image data, improving super-resolution results and enabling better resolution measurements.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) often produces images with lower resolution in the through-plane (slice-to-slice) direction compared to the in-plane (within-slice) directions. This anisotropy limits the ability to visualize fine details and can hinder subsequent image analysis tasks like registration and segmentation. To address this, super-resolution (SR) algorithms are employed to enhance the through-plane resolution. However, most SR methods rely on a "degeneration model" that simulates how a high-resolution (HR) image is converted into a low-resolution (LR) one. In MRI, this model is the slice selection profile (or slice profile), which is often unknown and can vary from the theoretically designed one due to acquisition imperfections. Existing SR algorithms either make simplifying assumptions about this profile (e.g., a Gaussian shape) or require external calibration data, which is not always feasible. This paper aims to estimate the slice profile directly from the MR image itself, without external information, to improve SR performance and provide a more accurate measure of image resolution.

Methods

The core idea of this paper is to leverage the self-similarity of MR images. The authors hypothesize that if an HR image patch is blurred by the correct slice profile and then downsampled, its resulting patch distribution should match the distribution of patches extracted directly from the LR through-plane direction of the MR volume.

To implement this, they propose a Generative Adversarial Network (GAN) framework. The generator in their GAN is designed to learn a "relative" slice profile (denoted as k). This relative profile is essentially the difference between the through-plane slice profile (pi) and the in-plane point spread function (ph). The generator takes an image patch, blurs it along one direction using the learned relative slice profile, and then downsamples it. The discriminator's role is to distinguish between these generated LR patches and real LR patches extracted from the through-plane direction.

The key novelty lies in formulating slice profile estimation as an internal patch distribution matching problem within a GAN. The generator learns to produce a slice profile that, when applied to HR patches, makes them statistically indistinguishable from real LR patches. This allows the slice profile to be learned end-to-end, directly from the image data, without needing explicit knowledge of the true HR image or the exact degeneration process. They also incorporate regularization terms to ensure the learned slice profile is physically plausible (e.g., positive and sums to one).

Datasets

The authors used simulated LR images derived from five subjects from the OASIS-3 dataset. These simulations involved blurring isotropic brain MR images with either Gaussian or rect functions of varying Full Width at Half Maximum (FWHM) and then downsampling them. No new datasets were introduced; instead, existing MR data was used to create controlled simulation scenarios for evaluation.

Key Insights and Ideas

  • Self-Similarity as a Key: The fundamental insight is that the statistical properties of image patches should be consistent across different orientations if the degradation process is correctly modeled. This allows for learning the unknown degradation kernel (slice profile) by matching distributions.
  • GANs for Implicit Kernel Learning: Using a GAN is a powerful way to learn complex, unknown transformations like the slice profile. The adversarial nature forces the generator to produce outputs that are not just visually similar but also statistically consistent with the target distribution.
  • "Relative" Slice Profile: Learning a "relative" slice profile (k) simplifies the problem. It allows the method to work even if the in-plane blurring (ph) is unknown, as long as the through-plane blurring (pi) can be expressed as ph convolved with k.
  • Slice Profile as a Resolution Metric: The ability to accurately estimate the slice profile also provides a direct way to measure the through-plane resolution of an image or an SR algorithm's output.

Key Results

  • The proposed method successfully estimated slice profiles in simulations, achieving good PSNR and SSIM values between degraded images and true HR images, even when the true slice profiles were complex (rect functions).
  • Incorporating the estimated slice profiles into an existing super-resolution algorithm (SMORE) led to improved PSNR and SSIM results, particularly when the slice thickness was smaller than the slice separation.
  • The method was demonstrated to be capable of measuring the through-plane resolution of super-resolved images, providing quantitative estimates of FWHM. For example, in one experiment, the estimated FWHM for Type 1 simulations was 1.9273 mm (SD=0.0583 mm), close to the true value.

Limitations and Future Work

  • Shape Flexibility: The authors acknowledge that their current generator architecture, designed for smoothness, struggles to learn highly flexible slice profile shapes (like rect functions) accurately. They suggest that future work could explore more flexible architectures or regularization strategies to address this.
  • Inconclusive Improvement in Certain Cases: While improvements were seen, the benefit of incorporating the estimated slice profile into SMORE was less pronounced or even counter-intuitive when slice thickness was larger than slice separation (leading to overlaps). This suggests that the interaction between the SR algorithm and the slice profile might be complex and warrants further investigation.
  • Future Directions: The authors plan to conduct more experiments with SMORE, including training for more iterations and testing other SR algorithms. They also suggest investigating better ways to regularize the slice profile to improve its flexibility and stability during training.

Summary updated

Abstract

To super-resolve the through-plane direction of a multi-slice 2D magnetic resonance (MR) image, its slice selection profile can be used as the degeneration model from high resolution (HR) to low resolution (LR) to create paired data when training a supervised algorithm. Existing super-resolution algorithms make assumptions about the slice selection profile since it is not readily known for a given image. In this work, we estimate a slice selection profile given a specific image by learning to match its internal patch distributions. Specifically, we assume that after applying the correct slice selection profile, the image patch distribution along HR in-plane directions should match the distribution along the LR through-plane direction. Therefore, we incorporate the estimation of a slice selection profile as part of learning a generator in a generative adversarial network (GAN). In this way, the slice selection profile can be learned without any external data. Our algorithm was tested using simulations from isotropic MR images, incorporated in a through-plane super-resolution algorithm to demonstrate its benefits, and also used as a tool to measure image resolution. Our code is at https://github.com/shuohan/espreso2.
BibTeX
@inproceedings{han2021mr,
  author = {Shuo Han and Samuel Remedios and Aaron Carass and Michael Schär and Jerry L. Prince},
  title = {{MR Slice Profile Estimation by Learning to Match Internal Patch Distributions}},
  booktitle = {IPMI},
  year = {2021},
  doi = {10.1007/978-3-030-78191-0\_9},
}
Figure