Learning the Sampling Pattern for MRI

Ferdia Sherry, Martin Benning, Juan Carlos de los Reyes, Martin J. Graves, Georg Maierhofer, Guy B. Williams, Carola-Bibiane Schönlieb, Matthias J. Ehrhardt

TL;DR

This paper introduces a novel bilevel learning approach to automatically discover optimal sparse sampling patterns for MRI, significantly reducing scan times without sacrificing image quality. By learning directly from data, their method adapts to specific imaging tasks and reconstruction methods, outperforming traditional fixed sampling strategies.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is a powerful diagnostic tool, but acquiring high-quality images often requires long scan times. This is problematic for patient comfort, can lead to motion artifacts, and limits patient throughput. Compressed sensing (CS) offers a theoretical framework to reconstruct images from fewer measurements than traditionally required, by exploiting the inherent sparsity of the image in some transform domain. However, the original CS theory often assumes uniform random sampling, which is not optimal for MRI. Variable density sampling patterns have shown promise, but designing these patterns optimally for specific MRI applications and reconstruction algorithms has been a challenge. This paper tackles the problem of learning these optimal, sparse sampling patterns directly from data, aiming to strike the best balance between acquisition speed and reconstructed image quality.

Methods

The core of this work is a bilevel optimization framework. The "upper level" problem learns the sampling pattern and the regularization parameter for the reconstruction. The "lower level" problem is the actual MRI reconstruction using a variational approach.

Here's a breakdown of the key methodological components:

  • Bilevel Optimization: This is the central idea. The authors frame the problem as a nested optimization:
  • Upper Level: Minimize a penalized empirical loss function over the sampling pattern parameters ($S$) and the regularization parameter ($\alpha$). This loss function measures the reconstruction quality across a training dataset. A sparsity penalty is applied to the sampling pattern itself to encourage fewer measurements.
  • Lower Level: For a given sampling pattern ($S$) and regularization parameter ($\alpha$), reconstruct the image ($u$) from undersampled measurements ($y$) by minimizing a variational energy functional. This functional balances fitting the measurements with enforcing image properties (e.g., sparsity in a transform domain).
  • Variational Regularization: The lower-level reconstruction uses a standard form of variational regularization: $\min_u \frac{1}{2} ||SFu - y||^2 + \alpha R(u)$. They consider various regularizers $R(u)$, including Total Variation (TV) and wavelet sparsity.
  • Continuous Parametrization of Sampling Patterns: To make the learning problem a continuous optimization problem, the sampling pattern is parametrized in a way that allows for learning arbitrary patterns, including scattered points, Cartesian lines, and potentially radial or spiral patterns.
  • Differentiable Reconstruction: A crucial aspect is the ability to differentiate the solution of the lower-level reconstruction problem with respect to the parameters of the sampling pattern. This is achieved by leveraging the implicit function theorem and solving linear systems involving the Hessian of the energy functional.
  • Optimization Algorithm: The bilevel problem is solved using the L-BFGS-B algorithm, a quasi-Newton method suitable for bound-constrained optimization. The gradients for this algorithm are computed using the chain rule and the derivatives of the lower-level solution.

Datasets

The paper uses several datasets for training and evaluation:

  • Brain Images:
    • T1-weighted 3D scans: Slices from these scans (192x192 resolution) are used for training and testing. A small training set of 7 slices is used, with a larger test set of 70 slices.
  • Simulated T2-weighted 3D scans (BrainWeb database): Slices from these (217x181 resolution) are used for some experiments, with a training set of 5 slices and a test set of 5 slices.
  • High-Resolution Phantom Scan: Slices from a high-resolution scan (1024x1024 resolution) are used for a high-resolution experiment, with a training set of 5 slices.

The paper does not introduce new benchmark datasets but uses existing MRI data (real and simulated) to demonstrate the effectiveness of their learned sampling patterns.

Key Insights and Ideas

  • Data-Driven Sampling is Superior: The most significant insight is that learning sampling patterns directly from representative data leads to demonstrably better reconstruction quality and efficiency compared to fixed, heuristic patterns (like uniform random or simple variable density).
  • Small Training Sets Suffice: Remarkably, the method can learn effective sampling patterns even with very small training sets (as few as 5-7 image-measurement pairs), suggesting good generalization capabilities.
  • Adaptability to Reconstruction: The framework is flexible and can learn sampling patterns tailored to specific reconstruction algorithms (e.g., TV vs. wavelet regularization), highlighting the interplay between sampling and reconstruction.
  • Sparsity vs. Quality Trade-off: The bilevel approach explicitly optimizes the trade-off between the sparsity of the sampling pattern (and thus acquisition time) and the quality of the reconstructed image, controlled by a sparsity parameter $\beta$.
  • Anisotropic Sampling: Learned patterns often exhibit anisotropic sampling densities, meaning they sample k-space differently along different directions, which is more efficient than isotropic variable density sampling.

Key Results

  • Significant Acceleration: Learned sampling patterns can achieve high reconstruction quality (e.g., SSIM of 0.914) while sampling only a fraction of k-space (e.g., 35% in one example).
  • Outperformance of Existing Methods: In comparisons, the learned patterns consistently outperformed uninformed variable density sampling and even some other data-adapted learning methods, especially in resolving fine image details.
  • Flexibility in Pattern Design: The method can learn various types of patterns, including scattered points and Cartesian lines, and can be extended to other trajectories like radial or spiral.
  • Robustness to Regularization: The learned patterns perform well across different lower-level regularization choices (TV, wavelet, H¹), though TV regularization generally yielded the best results.
  • High-Resolution Performance: The method scales to high-resolution images, demonstrating its potential for practical applications.

Limitations and Future Work

  • Computational Cost: The bilevel optimization is computationally intensive, requiring solving lower-level problems to high accuracy in each iteration. While warm-starting and efficient gradient computation help, further optimization of the computational implementation is suggested.
  • 2D vs. 3D MRI: The current work focuses on 2D MRI. Extending to 3D MRI is straightforward mathematically but requires significant optimization for computational efficiency and practical implementation of the learned patterns.
  • Free Patterns vs. Physical Constraints: The learned "free" patterns (scattered points) are not directly usable in practice due to physical constraints of MRI scanners. The authors suggest using these learned patterns to estimate target densities for existing practical sampling strategy generation methods.
  • Non-Convexity: The bilevel optimization problem is non-convex, and while the current initialization strategy (fixed sampling, learned regularization parameter) works well, further research into initialization and handling non-convexity is suggested.
  • Generalizing Regularizers: The framework could be extended to learn more general regularizers or even unroll the reconstruction algorithm into a neural network.
  • Stochastic Optimization: Investigating stochastic optimization methods could be beneficial given the data-parallel nature of the objective function.

Summary updated

Abstract

The discovery of the theory of compressed sensing brought the realisation that many inverse problems can be solved even when measurements are "incomplete". This is particularly interesting in magnetic resonance imaging (MRI), where long acquisition times can limit its use. In this work, we consider the problem of learning a sparse sampling pattern that can be used to optimally balance acquisition time versus quality of the reconstructed image. We use a supervised learning approach, making the assumption that our training data is representative enough of new data acquisitions. We demonstrate that this is indeed the case, even if the training data consists of just 7 training pairs of measurements and ground-truth images; with a training set of brain images of size 192 by 192, for instance, one of the learned patterns samples only 35% of k-space, however results in reconstructions with mean SSIM 0.914 on a test set of similar images. The proposed framework is general enough to learn arbitrary sampling patterns, including common patterns such as Cartesian, spiral and radial sampling.
BibTeX
@article{sherry2019learning,
  author = {Ferdia Sherry and Martin Benning and Juan Carlos de los Reyes and Martin J. Graves and Georg Maierhofer and Guy B. Williams and Carola-Bibiane Schönlieb and Matthias J. Ehrhardt},
  title = {{Learning the Sampling Pattern for MRI}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2019},
  url = {https://openreview.net/forum?id=bYo1KZvbny},
}
Figure