Deep Model-Based Architectures for Inverse Problems Under Mismatched Priors

Shirin Shoushtari, Jiaming Liu, Yuyang Hu, Ulugbek S. Kamilov

In the world of computational imaging, we often face "inverse problems"—trying to reconstruct a high-quality image from noisy, blurry, or incomplete measurements (like an MRI scan or a low-dose CT). Traditionally, we solved these using hand-crafted mathematical rules called "priors." Recently, Deep Model-Based Architectures (DMBAs) have taken over, replacing those rules with learned Convolutional Neural Networks (CNNs).

But there’s a catch: what happens when the CNN was trained on one type of data (say, cats and dogs) but is asked to reconstruct another (like a human brain)? This new paper from researchers at Washington University in St. Louis provides the first rigorous look at the "mismatched prior" problem.

TL;DR

This paper provides the first theoretical error bounds for deep learning-based image reconstruction when there is a "mismatch" between the learned prior and the actual data. The authors prove that these architectures are surprisingly robust, showing that even a mismatched deep prior often outperforms traditional hand-crafted methods like Total Variation (TV).

What Problem Does This Paper Address?

Most research into DMBAs (like Plug-and-Play or Deep Unfolding) assumes an idealized scenario: the CNN prior used during inference perfectly matches the distribution of the images being recovered.

In the real world, this is rarely true. Mismatches happen because of:

  1. Distribution Shift: You trained your model on natural images, but your test data is medical MRI scans.
  2. Approximation: You used a CNN to approximate a complex statistical estimator (like a MAP or MMSE estimator) to save on computation time.

Until now, we didn't have a clear mathematical understanding of how these discrepancies affect the final image. This paper fills that gap by asking: How much error does a mismatched prior actually introduce, and can we guarantee the algorithm will still converge?

Methods

The authors focus on an architecture called SD-RED (Steepest Descent variant of Regularization by Denoising).

  • Theoretical Framework: They use monotone operator theory to analyze the algorithm. By treating the CNN as a mathematical operator, they derived explicit error bounds.
  • The "Mismatch" Parameter: They introduce a constant ($\epsilon$) that quantifies the distance between the "true" desired prior and the "mismatched" CNN prior actually being used.
  • Novelty: Unlike previous work that assumes the prior is perfect ($\epsilon = 0$), this paper provides bounds that scale with $\epsilon$, the regularization strength ($\sigma$), and the step size ($\gamma$).
  • Statistical Bridge: They provide a theorem linking the density ratio of two probability distributions to the distance between their corresponding estimators, effectively bridging abstract math with real-world data statistics.

Datasets

The study evaluated performance across several diverse datasets to simulate different levels of mismatch:

  • Natural Images: Training on general grayscale images [69] and testing on the standard Set11 benchmark.
  • Medical Imaging: Brain MRI images from the IXI dataset [70] and CT images from the Mayo Clinic "low dose CT grand challenge" [71].
  • fastMRI: Used for parallel MRI experiments, including Knee, Brain, and AXT1/AXT2 modalities [76].

Key Insights and Ideas

  • The Error is Bounded: The most important "aha" moment is that the reconstruction error doesn't explode when the prior is wrong. Instead, the distance to the true solution is proportional to the level of mismatch ($\epsilon$).
  • Deep Learning is Resilient: Even "severely mismatched" priors (e.g., using a CNN trained on natural images to reconstruct an MRI) are remarkably effective. They capture low-level features (edges, textures) that are universal enough to be useful across domains.
  • CNNs as Fast Approximators: The authors showed that a CNN can be trained to mimic traditional Total Variation (TV). This "TV-Approx" prior yields nearly identical results to the exact mathematical TV solver, suggesting we can use deep learning to speed up classical algorithms without losing accuracy.
  • The Sampling Factor: As the sampling ratio increases (i.e., you have more data), the algorithm becomes less vulnerable to distribution shifts in the prior.

Key Results

  • Mismatched MRI: In a 10% subsampled MRI reconstruction task, a prior trained on MRI data achieved 32.39 dB (PSNR). A prior trained on natural images—a significant mismatch—still achieved 31.08 dB. Crucially, both significantly outperformed the traditional TV method (28.89 dB).
  • Parallel MRI: In tests on the fastMRI dataset, mismatched priors (e.g., using a "Knee" prior on "Brain" data) consistently outperformed traditional TV reconstruction across different acceleration factors (2x and 4x).
  • Convergence: The numerical results perfectly mirrored the theoretical predictions: as the regularization parameters ($\tau$ and $\sigma$) increased, the "gap" or error caused by the mismatched prior also increased.

Limitations and Future Work

  • Architecture Specificity: The theoretical analysis is primarily tailored to the SD-RED iteration. While the authors suggest the results can be extended to other DMBAs (like PnP-ADMM), those specific proofs aren't detailed here.
  • Characterizing Mismatch: The current model uses a simple distance bound ($\epsilon$) to quantify mismatch. Future work could explore more nuanced ways to characterize how distributions differ (e.g., using Optimal Transport or Wasserstein distances).
  • Active Adaptation: The authors suggest future research could focus on developing methods that "reduce" the influence of the mismatch during the inference process itself, perhaps through test-time adaptation.

Summary updated

Abstract

There is a growing interest in deep model-based architectures (DMBAs) for solving imaging inverse problems by combining physical measurement models and learned image priors specified using convolutional neural nets (CNNs). For example, well-known frameworks for systematically designing DMBAs include plug-and-play priors (PnP), deep unfolding (DU), and deep equilibrium models (DEQ). While the empirical performance and theoretical properties of DMBAs have been widely investigated, the existing work in the area has primarily focused on their performance when the desired image prior is known exactly. This work addresses the gap in the prior work by providing new theoretical and numerical insights into DMBAs under mismatched CNN priors. Mismatched priors arise naturally when there is a distribution shift between training and testing data, for example, due to test images being from a different distribution than images used for training the CNN prior. They also arise when the CNN prior used for inference is an approximation of some desired statistical estimator (MAP or MMSE). Our theoretical analysis provides explicit error bounds on the solution due to the mismatched CNN priors under a set of clearly specified assumptions. Our numerical results compare the empirical performance of DMBAs under realistic distribution shifts and approximate statistical estimators.
BibTeX
@article{shoushtari2022deep,
  author = {Shirin Shoushtari and Jiaming Liu and Yuyang Hu and Ulugbek S. Kamilov},
  title = {{Deep Model-Based Architectures for Inverse Problems Under Mismatched Priors}},
  journal = {IEEE Journal on Selected Areas in Information Theory},
  year = {2022},
  url = {https://openreview.net/forum?id=iSBOyj4Y3P},
}

Loading related content...

Figure