Accelerating MRI Uncertainty Estimation with Mask-based Bayesian Neural Network

Zehuan Zhang, Matej Genci, Hongxiang Fan, Andreas Wetscherek, Wayne Luk

This paper presents a specialized algorithm-hardware co-design to bring real-time uncertainty estimation to Magnetic Resonance Imaging (MRI) analysis, specifically for adaptive radiotherapy. By converting a standard deep neural network into a mask-based Bayesian Neural Network and deploying it on a custom FPGA accelerator, the authors achieve significant speedups and energy savings over traditional CPU and GPU implementations.

TL;DR

The authors propose uIVIM-NET, a hardware-efficient Bayesian Neural Network that provides critical uncertainty estimates for MRI analysis in real-time. By using fixed masks instead of runtime random sampling and a custom FPGA accelerator, they achieve a 32.5x speedup over CPUs and a 7.5x speedup over GPUs while drastically reducing power consumption.

What Problem Does This Paper Address?

In the fight against cancer, adaptive radiotherapy is a game-changer. It uses real-time MRI to adjust radiation doses based on the exact position and state of a tumor just before treatment. A popular model for this is IVIM-NET, which analyzes Intravoxel Incoherent Motion (IVIM) to distinguish between water diffusion and blood perfusion in tissues.

However, IVIM-NET has two major flaws for clinical use:

  1. Lack of Uncertainty: It provides "point estimates" without telling clinicians how confident it is. In medical settings, an overconfident but wrong prediction can be dangerous.
  2. Computational Cost: Traditional ways to add uncertainty (like Bayesian Neural Networks) require multiple forward passes and runtime random sampling, making them too slow and power-hungry for the real-time requirements of a treatment room.

Methods

The authors tackle these issues through a two-pronged "co-optimization" flow:

1. The Algorithm: From DNN to Mask-based BayesNN

Instead of using standard Monte Carlo (MC) Dropout—which requires generating random numbers at runtime—the authors use Masksembles. This technique uses a set of pre-defined, fixed masks to simulate an ensemble of models.

![Fig.

  1. Conversion from IVIM-NET to uIVIM-NET](/figures/art_3fa7a21fbcfb4a519d31e2bef8d012bf)

As shown in Figure 2, the dropout layers in the original IVIM-NET are replaced with these fixed masks. This makes the "randomness" deterministic at runtime, which is much easier for hardware to handle.

2. The Hardware: Custom FPGA Accelerator

The authors designed a parallel, pipelined accelerator on a Xilinx VU13P FPGA. Key optimizations include:

  • Mask-Zero Skipping: Since the masks are fixed and known beforehand, the hardware simply skips storing or calculating weights that are zeroed out. This eliminates the need for complex runtime sampling modules.
  • Batch-level Operation Reordering: In a typical setup, you might load weights for every single voxel (sampling-level). The authors reordered this to process a whole batch of voxels with one mask before switching weights. This reduces weight-loading operations by a factor equal to the batch size, slashing power consumption.

![Fig.

  1. Sampling-level scheme vs. Batch-level scheme](/figures/art_e32b95a888e345d89fba17fbb25d1efa)

Datasets

Because ground-truth uncertainty is impossible to "measure" in real patients, the authors used synthetic datasets generated from the physical IVIM equations.

  • Size: 10,000 data points per set.
  • Noise Levels: Five different Signal-to-Noise Ratios (SNR 5, 15, 20, 30, and 50) to simulate various imaging conditions.
  • Validation: They also referenced a published IVIM dataset to ensure their accelerator could handle realistic voxel counts (up to 104 b-values).

Key Insights and Ideas

  • Deterministic Randomness is a Hardware Gift: By moving from runtime sampling (MC Dropout) to fixed masks (Masksembles), the authors turned a stochastic problem into a sparse, deterministic one. This allowed them to "bake" the sparsity into the hardware via mask-zero skipping.
  • Memory Bandwidth is the Real Bottleneck: The move to a "batch-level" scheme (Figure 5) highlights that in Bayesian inference, the energy cost isn't just in the math, but in moving the weights. Reordering operations to maximize weight reuse is critical for medical devices that need to stay cool and efficient.

Key Results

The co-design approach met the real-time requirement of 0.8ms per batch for adaptive radiotherapy, clocking in at 0.28ms.

Metric CPU (Xeon Silver) GPU (GTX 1080 Ti) Ours (FPGA)
Latency (ms/Batch) 9.1 2.1 0.28
Power (W) 30 54 11.78
Energy Cost (mJ/Batch) 273 113.4 3.3
  • Speedup: 32.5x vs. CPU; 7.5x vs. GPU.
  • Energy Efficiency: 82.8x more efficient than CPU; 34.4x more than GPU.
  • Uncertainty Quality: The model correctly showed higher uncertainty (variance) as the input noise increased, providing a reliable signal for clinicians.

Limitations and Future Work

  • DSP Constraints: The authors noted that the primary limiting factor for further speedup is the number of available Digital Signal Processing (DSP) slices on the FPGA.
  • Synthetic Data: While necessary for ground-truth uncertainty, the model still needs validation on a wider array of real-world clinical datasets.
  • Expansion: The authors suggest this flow could be extended beyond MRI to other high-stakes fields like autonomous driving or robotics, where real-time uncertainty is equally vital.

Summary updated

Abstract

Accurate and reliable Magnetic Resonance Imaging (MRI) analysis is particularly important for adaptive radiotherapy, a recent medical advance capable of improving cancer diagnosis and treatment. Recent studies have shown that IVIM-NET, a deep neural network (DNN), can achieve high accuracy in MRI analysis, indicating the potential of deep learning to enhance diagnostic capabilities in healthcare. However, IVIM-NET does not provide calibrated uncertainty information needed for reliable and trustworthy predictions in healthcare. Moreover, the expensive computation and memory demands of IVIM-NET reduce hardware performance, hindering widespread adoption in realistic scenarios. To address these challenges, this paper proposes an algorithm-hardware co-optimization flow for high-performance and reliable MRI analysis. At the algorithm level, a transformation design flow is introduced to convert IVIM-NET to a mask-based Bayesian Neural Network (BayesNN), facilitating reliable and efficient uncertainty estimation. At the hardware level, we propose an FPGA-based accelerator with several hardware optimizations, such as mask-zero skipping and operation reordering. Experimental results demonstrate that our co-design approach can satisfy the uncertainty requirements of MRI analysis, while achieving 7.5 times and 32.5 times speedup on an Xilinx VU13P FPGA compared to GPU and CPU implementations with reduced power consumption.
BibTeX
@inproceedings{zhang2024accelerating,
  author = {Zehuan Zhang and Matej Genci and Hongxiang Fan and Andreas Wetscherek and Wayne Luk},
  title = {{Accelerating MRI Uncertainty Estimation with Mask-based Bayesian Neural Network}},
  booktitle = {Asap},
  year = {2024},
  doi = {10.1109/asap61560.2024.00030},
}
Figure