MRI Image Reconstruction via Learning Optimization Using Neural ODEs

Eric Z. Chen, Terrence Chen, Shanhui Sun

TL;DR

This paper frames MRI image reconstruction as a continuous optimization problem, modeling the reconstruction trajectory using Neural Ordinary Differential Equations (ODEs). By designing neural networks that mimic and learn the behavior of advanced ODE solvers (like Runge-Kutta), the authors achieve highly accurate MRI reconstructions using a fraction of the parameters required by standard deep learning models.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is notoriously slow. To speed up the process, technicians often acquire "undersampled" data (capturing fewer data points in the frequency domain, or k-space). However, this undersampling creates aliasing artifacts, resulting in blurry or distorted images.

Recovering a clean image from undersampled data is an inverse problem. Traditionally, deep learning tackles this by using architectures like UNets or cascaded CNNs to map the messy data to a clean image, often mimicking discrete, step-by-step iterative optimization algorithms. The authors of this paper ask a fundamental question: instead of treating optimization as a series of discrete steps, what if we model the reconstruction process as a continuous flow over time?

Methods

The authors propose ReconODE, a framework that treats the MRI reconstruction process as a continuous dynamic trajectory governed by an ODE. The initial state is the undersampled image, and the final state is the fully reconstructed image.

They explore two main approaches to solving this ODE:

  1. Fixed Solvers (ReconODE-FT / ReconODE-FA): A Convolutional Neural Network (CNN) is used to learn the "dynamics" (how the image changes over time). To get the final image, they use standard, off-the-shelf mathematical ODE solvers like Euler, RK2, or RK4 to integrate the CNN's output. They train these using either standard backpropagation through the solver (FT) or the memory-efficient adjoint sensitivity method (FA).
  2. Learned Solvers (ReconODE-LT): This is the paper's novel twist. Instead of using rigid mathematical formulas for the solvers, they build neural network blocks that mimic the structural steps of Euler and Runge-Kutta solvers. The network implicitly learns the optimal coefficients and step sizes. They also replace standard weighted sums in the solver equations with an attention mechanism to better capture multi-scale features.

To handle the high memory cost of training these complex learned solvers, they apply gradient checkpointing, which trades a tiny bit of compute time for massive memory savings.

Datasets

  • fastMRI Knee Dataset (Single-Coil): A standard benchmark dataset in the field.
  • Splits: 34,742 2D slices for training and 7,135 slices for validation.
  • Task: Reconstructing fully sampled images from data undersampled at acceleration factors of 4x and 8x.

Key Insights and Ideas

  • Continuous > Discrete: By modeling the optimization trajectory continuously via ODEs, the network becomes incredibly parameter-efficient. It achieves comparable or better results than traditional models while using significantly fewer parameters.
  • Learning the Math Pays Off: While using fixed mathematical ODE solvers works, allowing the neural network to learn the solver's parameters (ReconODE-LT) yields noticeably sharper images and better quantitative metrics.
  • Backprop Beats the Adjoint Method: Neural ODEs are famous for using the "adjoint method" to save memory during training. However, the authors found that standard backpropagation through the solver operations actually resulted in more accurate reconstructions.
  • Attention is Useful in Solvers: Replacing the static weighted sums of traditional Runge-Kutta solvers with a learned attention module successfully helped the network recover more high-frequency details.

Key Results

  • Extreme Parameter Efficiency: The best-performing model (ReconODE-LT-RK4) uses only 0.15 million parameters. For context, a standard UNet baseline uses 3.35 million parameters (about 22x larger).
  • Top Accuracy: Despite its small size, ReconODE-LT-RK4 outperformed UNet, KIKI-Net, and a cascaded CNN (D5C5).
  • At 4x acceleration, it achieved a PSNR of 32.39 and SSIM of 0.7333.
  • At 8x acceleration, it achieved a PSNR of 30.27 and SSIM of 0.6617.
  • Memory Savings: Using gradient checkpointing on the ReconODE-LT-RK4 model reduced GPU memory usage by 73% (from 13.6GB to 3.7GB) while only adding 0.13 seconds to the training time per step.

Limitations and Future Work

  • Evaluation Limitations: Because the fastMRI test leaderboard was closed at the time of writing, the authors could only report results on the validation set.
  • Scale: The proposed models are very small ($\le$ 0.15M parameters). State-of-the-art models on the fastMRI leaderboard often use massive ensembles and 20M–30M parameters. Therefore, direct performance comparisons against the absolute top-tier leaderboard models aren't 1-to-1.
  • Future Work: The authors view this as a foundational new framework. They suggest that scaling up the network size for the ODE dynamics and exploring even more complex learned solvers will likely yield massive performance boosts in future research.

Summary updated

Abstract

We propose to formulate MRI image reconstruction as an optimization problem and model the optimization trajectory as a dynamic process using ordinary differential equations (ODEs). We model the dynamics in ODE with a neural network and solve the desired ODE with the off-the-shelf (fixed) solver to obtain reconstructed images. We extend this model and incorporate the knowledge of off-the-shelf ODE solvers into the network design (learned solvers). We investigate several models based on three ODE solvers and compare models with fixed solvers and learned solvers. Our models achieve better reconstruction results and are more parameter efficient than other popular methods such as UNet and cascaded CNN. We introduce a new way of tackling the MRI reconstruction problem by modeling the continuous optimization dynamics using neural ODEs.
BibTeX
@inproceedings{chen2020mri,
  author = {Eric Z. Chen and Terrence Chen and Shanhui Sun},
  title = {{MRI Image Reconstruction via Learning Optimization Using Neural ODEs}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  url = {https://openreview.net/forum?id=Y73u4B6utd},
}

Loading related content...

Figure