Micro-Networks for Robust MR-Guided Low Count PET Imaging

Casper da Costa‐Luis, Andrew J. ReaderIEEE Transactions on Radiation and Plasma Medical Sciences 2020 · DOI

TL;DR

This paper introduces "micro-networks" (µ-nets), a novel, lightweight deep learning approach for improving the quality of low-count PET images. By leveraging MR guidance and requiring minimal training data, these µ-nets significantly reduce noise and artifacts while enhancing resolution, outperforming traditional methods like post-smoothing and even larger deep learning models like U-Nets in low-data scenarios.

What Problem Does This Paper Address?

Positron Emission Tomography (PET) imaging is crucial for diagnosing and monitoring diseases, but acquiring high-quality images often requires high doses of radioactive tracers or long scan times. When reducing tracer dose or scan time (resulting in "low count" data), image noise becomes a significant problem. Traditional methods to combat this noise, such as post-smoothing (PS) and regularization, often degrade image resolution and introduce bias. While anatomical information from Magnetic Resonance Imaging (MR) can help, effectively integrating this information into PET reconstruction, especially with limited training data, remains a challenge. Existing deep learning approaches, particularly Convolutional Neural Networks (CNNs), typically demand vast amounts of training data and have been underutilized for PET image quality enhancement, often focusing on classification or segmentation tasks.

Methods

The core innovation is the "micro-network" (µ-net), a 3D CNN designed for robustness with limited training data. Key features include:

  • Low Complexity: The µ-net has a significantly smaller parameter space compared to larger architectures like U-Nets. This is achieved through a shallow architecture (three layers) with a moderate number of kernels per layer.
  • Multi-Modal Input: The network accepts multiple input channels, including:
  • Low-count PET reconstructions (with and without resolution modeling - RM).
  • MR (T1-weighted) images.
  • Non-local means (NLM) filtered PET images, guided by MR. This allows the network to leverage complementary information from different sources.

  • 3D Convolution: The network operates on full 3D volumes, enabling it to utilize spatial context across slices, which is crucial for correcting partial volume effects (PVEs) and distinguishing signal from noise.

  • Sigmoid Activation Functions: Unlike common ReLU activations, sigmoids are used in the hidden layers. This allows information to propagate through the network without discarding potentially useful negative values, and acts as a form of non-linear kernel sensitivity control.
  • ELU Final Layer: An Exponential Linear Unit (ELU) is used in the final layer to provide a soft, lower-bound constraint, avoiding issues with strict non-negativity that can hinder optimization.
  • Robust Training: The network is trained to minimize Normalized Root Mean Squared Error (NRMSE) against a target (either ground truth or standard count reconstruction). Training uses multiple noise realizations of the same data to improve robustness.

For comparison, the authors also trained and evaluated a modified U-Net architecture, which is known for its effectiveness in image segmentation and reconstruction but typically requires more data.

Datasets

The study utilized two primary types of datasets:

  • Simulated Data: MR-based BrainWeb segmentations of 20 subjects were modified to create realistic PET phantoms. These phantoms were then used to generate simulated PET data at various count levels (3M, 30M, and 300M counts) with realistic noise properties, including scatter and randoms. The "ground truth" was a high-quality, high-count phantom.
  • Real Patient Data: Ten [18F]FDG PET head scans were acquired. List-mode data was then resampled to create low-count datasets (3M, 30M, and 300M counts) for each patient. Corresponding MPRAGE T1 MR reconstructions were also used.

The simulated data was used for training and initial evaluation, while both simulated and real patient data were used for final testing and comparison.

Key Insights and Ideas

  • The Power of "Micro": The paper demonstrates that a small, computationally efficient network (µ-net) can outperform larger, more complex models (like U-Nets) when training data is scarce. This challenges the common assumption that deeper and wider networks are always better.
  • Overfitting is a Major Hurdle: The U-Net, despite its potential, severely overfit the limited training data, leading to poor performance on unseen test data. This highlights the critical importance of network architecture choice in low-data regimes.
  • MR Guidance is Key: Integrating MR information as an input channel significantly improves PET image quality, providing anatomical context that helps the network denoise and reconstruct better images.
  • RM vs. T1 Importance: Interestingly, the results suggest that resolution modeling (RM) in PET might be more critical for quality improvement than T1 information alone, at least for the noise levels tested.
  • Multi-Realization Training: Training with multiple noise realizations of the same data, rather than just one, improves network robustness without a significant increase in computational cost.

Key Results

  • NRMSE Reduction: The µ-net achieved a 36% lower NRMSE compared to standard MLEM reconstruction on simulated low-count (30M) data, when trained to produce standard (300M) count reconstructions.
  • Comparison to Traditional Methods: This improvement was significantly better than optimized post-smoothing (25% NRMSE reduction) and comparable to using both RM and optimized PS (26% NRMSE reduction).
  • Robustness to Overfitting: The µ-net showed excellent generalization, with validation and training loss curves closely tracking each other, unlike the U-Net which overfit significantly.
  • Visual Quality: Resultant images from the µ-net exhibited lower noise, reduced ringing and partial volume effects, sharper edges, and improved resolution compared to conventional MLEM.
  • Real Data Performance: Similar improvements were observed for low-count real patient datasets.
  • Architecture Choice: The study found that a network with 32 kernels per layer was sufficient, avoiding the need for excessive complexity.

Limitations and Future Work

  • Limited Training Data: The primary focus is on scenarios with very limited training data. The performance of the µ-net with larger datasets is not explored.
  • Mismatched Noise Levels: The current work trains and tests on similar noise levels. Future work should investigate the impact of testing on different noise levels than those used for training.
  • Single Architecture per Case: The current approach retrains a network for each specific noise level and iteration scenario. Future work could explore a single architecture that can compensate for noise and artifacts across various conditions.
  • GAN Integration: The authors suggest combining their µ-net approach with Generative Adversarial Networks (GANs) for data augmentation, which could further enhance performance and robustness.
  • Extending to Other Modalities: The framework could be extended to include joint modality processing, such as PET-guided MR reconstruction or even PET prediction based on MR.

Summary updated

Abstract

Noise suppression is particularly important in low count positron emission tomography (PET) imaging. Post-smoothing (PS) and regularization methods which aim to reduce noise also tend to reduce resolution and introduce bias. Alternatively, anatomical information from another modality such as magnetic resonance (MR) imaging can be used to improve image quality. Convolutional neural networks (CNNs) are particularly well suited to such joint image processing, but usually require large amounts of training data and have mostly been applied outside the field of medical imaging or focus on classification and segmentation, leaving PET image quality improvement relatively understudied. This article proposes the use of a relatively low-complexity CNN (micro-net) as a post-reconstruction MR-guided image processing step to reduce noise and reconstruction artefacts while also improving resolution in low count PET scans. The CNN is designed to be fully 3-D, robust to very limited amounts of training data, and to accept multiple inputs (including competitive denoising methods). Application of the proposed CNN on simulated low (30 M) count data (trained to produce standard (300 M) count reconstructions) results in a 36% lower normalized root mean squared error (NRMSE, calculated over ten realizations against the ground truth) compared to maximum-likelihood expectation maximization (MLEM) used in clinical practice. In contrast, a decrease of only 25% in NRMSE is obtained when an optimized (using knowledge of the ground truth) PS is performed. A 26% NRMSE decrease is obtained with both RM and optimized PS. Similar improvement is also observed for low count real patient datasets. Overfitting to training data is demonstrated to occur as the network size is increased. In an extreme case, a U-net (which produces better predictions for training data) is shown to completely fail on test data due to overfitting to this case of very limited training data. Meanwhile, the resultant images from the proposed CNN (which has low training data requirements) have lower noise, reduced ringing, and partial volume effects, as well as sharper edges and improved resolution compared to conventional MLEM.
BibTeX
@article{costaluis2020micro,
  author = {Casper da Costa‐Luis and Andrew J. Reader},
  title = {{Micro-Networks for Robust MR-Guided Low Count PET Imaging}},
  journal = {IEEE Transactions on Radiation and Plasma Medical Sciences},
  year = {2020},
  doi = {10.1109/trpms.2020.2986414},
}
Figure