Stabilizing Deep Tomographic Reconstruction Networks

Weiwen Wu, Dianlin Hu, Hengyong Yu, Hongming Shan, Shaoyu Wang, Wenxiang Cong, Chuang Niu, Pingkun Yan, Vince Vardhanabhuti, Ge Wang

TL;DR

Deep learning methods for tomographic reconstruction (like CT and MRI) are powerful but suffer from instabilities, producing artifacts or missing details. This paper introduces ACID, a novel framework that combines deep learning with compressed sensing principles to create stable and accurate reconstructions, overcoming these limitations and paving the way for clinical adoption.

What Problem Does This Paper Address?

Tomographic imaging, crucial for medical diagnostics (CT, MRI), has seen a surge in deep learning-based reconstruction methods. These data-driven approaches promise faster and more accurate results. However, a recent study highlighted significant instabilities in these deep networks. These instabilities manifest as:

  1. Artifacts from tiny perturbations: Small, imperceptible changes in input data can lead to severe image artifacts, potentially causing misinterpretations (false positives).
  2. Missing small features: Crucial details, like small tumors, might be entirely absent in the reconstructed image (false negatives).
  3. Performance degradation with more data: Counterintuitively, providing more input data can sometimes worsen the reconstruction quality.

These issues stem from a lack of "kernel awareness" in deep networks, a concept that traditional compressed sensing (CS) methods inherently possess. CS methods, while more stable, can be less flexible. The core problem is how to leverage the power of deep learning while ensuring the robustness and reliability required for clinical applications.

Methods

The paper proposes the Analytic Compressed Iterative Deep (ACID) framework. ACID is a hybrid approach that synergizes three key components:

  1. Deep Reconstruction Network (Φ): A pre-trained deep neural network capable of learning complex data-driven priors from large datasets. This network provides the initial reconstruction.
  2. Compressed Sensing (CS) Module (Θ): This module enforces sparsity and "kernel awareness" on the reconstructed image. It acts as a regularizer, preventing the instabilities seen in pure deep learning methods. The paper mentions this can be a standard CS algorithm or a network-based implementation.
  3. Iterative Refinement: ACID employs an iterative process to minimize the "data residual"—the difference between the measured data and the data synthesized from the current reconstruction. This residual is fed back to the deep network to refine the image, ensuring consistency with the measurements.

The framework essentially finds a solution that lies at the intersection of data-driven solutions, sparse solutions, and solutions consistent with the measurements. The theoretical analysis relies on a Bounded Relative Error Norm (BREN) property, a specific form of Lipschitz continuity, to demonstrate convergence.

Datasets

The paper uses datasets and naming conventions from a prior study [9] for comparability. Specific datasets mentioned include:

  • CT: Ellipse phantoms with structural changes (cases C1-C7), including text and bird icons.
  • MRI: Brain MRI slices (cases M1-M12), with added text and structural changes.
  • Publicly available datasets: MGH-USC HCP public dataset for MRI training.

The paper also mentions using the FBP method to generate training data for some networks.

Key Insights and Ideas

  • Kernel Awareness is Crucial: The paper strongly emphasizes that the lack of "kernel awareness" is the root cause of deep learning instabilities in image reconstruction. CS methods inherently possess this, which ACID aims to integrate.
  • Hybrid Approach Outperforms Pure Methods: ACID demonstrates that combining the data-driven power of deep learning with the robustness of CS principles yields superior results compared to either method alone.
  • Stability is Achievable: The proposed ACID framework successfully stabilizes previously unstable deep reconstruction networks, making them reliable for clinical use.
  • BREN Property as a Theoretical Foundation: The Bounded Relative Error Norm (BREN) property is introduced as a key assumption for analyzing the convergence of ACID, providing a theoretical basis for its stability.
  • Resilience to Adversarial Attacks: ACID shows remarkable resilience against adversarial attacks, a significant concern for deep learning models in sensitive applications.

Key Results

  • Superior Image Quality: ACID consistently achieved higher Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) scores compared to baseline methods (Ell-50, DAGAN, AUTOMAP, and CS-based methods) across various scenarios. For example, in one CT case, ACID improved PSNR by over 8 dB compared to AUTOMAP.
  • Elimination of Instabilities: ACID effectively addressed the three reported instabilities: it eliminated strong artifacts from tiny perturbations, recovered missing small features, and maintained or improved performance with more input data.
  • Adversarial Attack Resilience: ACID reconstructions were significantly more robust to adversarial attacks than pure deep learning methods, preserving critical details and edges.
  • Computational Efficiency: While iterative, ACID was shown to be faster than some dictionary learning-based methods, with one example showing ACID taking only 9.2 seconds compared to 606.3 seconds for DLMRI.
  • Stability Against Noise: Empirical results demonstrated ACID's local stability with respect to noise, with output-to-input variation ratios significantly lower than those of other methods.

Limitations and Future Work

  • Theoretical Rigor: The authors acknowledge that their theoretical analysis, particularly the convergence analysis, relies on approximations and is not fully rigorous, especially given the nascent state of non-convex optimization theory.
  • BREN Property Assumption: The convergence analysis hinges on the assumption that the reconstruction network satisfies the BREN property. While empirically verified, a more general theoretical guarantee would be beneficial.
  • Generalizability of ACID: While ACID is presented as an exemplary embodiment, the authors suggest that other hybrid reconstruction schemes could be investigated.
  • Clinical Translation: The paper anticipates that this integrative approach will help promote the development and translation of deep tomographic image reconstruction networks into clinical applications, implying that further validation and integration are needed.

Summary updated

Abstract

Tomographic image reconstruction with deep learning is an emerging field, but a recent landmark study reveals that several deep reconstruction networks are unstable for computed tomography (CT) and magnetic resonance imaging (MRI). Specifically, three kinds of instabilities were reported: (1) strong image artefacts from tiny perturbations, (2) small features missing in a deeply reconstructed image, and (3) decreased imaging performance with increased input data. On the other hand, compressed sensing (CS) inspired reconstruction methods do not suffer from these instabilities because of their built-in kernel awareness. For deep reconstruction to realize its full potential and become a mainstream approach for tomographic imaging, it is thus critically important to meet this challenge by stabilizing deep reconstruction networks. Here we propose an Analytic Compressed Iterative Deep (ACID) framework to address this challenge. ACID synergizes a deep reconstruction network trained on big data, kernel awareness from CS-inspired processing, and iterative refinement to minimize the data residual relative to real measurement. Our study demonstrates that the deep reconstruction using ACID is accurate and stable, and sheds light on the converging mechanism of the ACID iteration under a Bounded Relative Error Norm (BREN) condition. In particular, the study shows that ACID-based reconstruction is resilient against adversarial attacks, superior to classic sparsity-regularized reconstruction alone, and eliminates the three kinds of instabilities. We anticipate that this integrative data-driven approach will help promote development and translation of deep tomographic image reconstruction networks into clinical applications.
BibTeX
@misc{wu2020stabilizing,
  author = {Weiwen Wu and Dianlin Hu and Hengyong Yu and Hongming Shan and Shaoyu Wang and Wenxiang Cong and Chuang Niu and Pingkun Yan and Vince Vardhanabhuti and Ge Wang},
  title = {{Stabilizing Deep Tomographic Reconstruction Networks}},
  year = {2020},
  eprint = {2008.01846},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2008.01846},
}
Figure