Provable Convergence of Plug-and-Play Priors With MMSE Denoisers

Xiaojian Xu, Yu Sun, Jiaming Liu, Brendt Wohlberg, Ulugbek S. KamilovIEEE Signal Processing Letters 2020 · OpenReview

TL;DR

This paper bridges a critical gap in Plug-and-Play (PnP) image reconstruction by providing the first theoretical convergence guarantees for PnP algorithms using Minimum Mean Squared Error (MMSE) denoisers. The authors show that a PnP variant of the Iterative Shrinkage-Thresholding Algorithm (ISTA) with MMSE denoisers converges to a stationary point, even for non-convex problems, and demonstrate this with both exact and learned MMSE denoisers.

What Problem Does This Paper Address?

Regularized image reconstruction aims to recover an unknown signal from noisy measurements, often by solving an optimization problem that balances data fidelity with a prior model. Plug-and-Play (PnP) is a powerful framework that replaces the explicit prior term with a pre-trained denoiser. While PnP has been successfully analyzed for Maximum A Posteriori (MAP) denoisers, there's been a lack of theoretical understanding for Minimum Mean Squared Error (MMSE) denoisers. MMSE denoisers are desirable for their optimality in terms of signal-to-noise ratio (SNR), but they are often non-expansive and computationally challenging, making their theoretical analysis in PnP settings difficult. This paper tackles this challenge by providing the first convergence analysis for PnP algorithms employing MMSE denoisers.

Methods

The core contribution is the theoretical convergence analysis of a PnP variant of the Iterative Shrinkage-Thresholding Algorithm (PnP-ISTA) when using MMSE denoisers. The authors establish that under mild assumptions (non-degenerate prior, Lipschitz continuous gradient for the data fidelity term, and a bounded objective function), the iterates generated by PnP-ISTA with an MMSE denoiser converge to a stationary point of a global cost function. This analysis is achieved by reformulating the MMSE denoiser as a proximal operator of a specific regularizer, allowing the application of majorization-minimization (MM) strategies. Crucially, this analysis does not require the data fidelity term or the regularizer to be convex, nor does it assume the denoiser is non-expansive.

Datasets

Not discussed in the paper. The paper focuses on theoretical analysis and numerical evaluation using synthetic data generated for sparse signal recovery.

Key Insights and Ideas

  • Bridging the MMSE-ISTA Gap: The paper provides the first theoretical convergence proof for PnP-ISTA using MMSE denoisers, a significant advancement given the optimality of MMSE estimators.
  • MMSE as a Proximal Operator: A key insight is the formal connection established between MMSE estimation and the proximal operator of a specific, potentially non-convex, regularizer. This allows existing PnP convergence tools to be applied.
  • Convergence without Convexity/Nonexpansiveness: The theoretical result is notable for not requiring convexity of the objective function or nonexpansiveness of the denoiser, which are common limitations in other PnP analyses. This broadens the applicability of PnP-ISTA.
  • Learned Denoisers as Practical MMSE Approximations: The work highlights that approximate MMSE denoisers, such as those learned by deep neural networks (like DnCNN), can effectively substitute for exact MMSE denoisers in PnP-ISTA, achieving comparable performance.
  • Suboptimality of ISTA for MMSE: The numerical results suggest that while PnP-ISTA with MMSE denoisers shows good convergence, it may still be suboptimal compared to algorithms like GAMP, potentially due to ISTA's assumption of Gaussian noise at each iteration, which doesn't hold for MMSE denoisers.

Key Results

  • Theoretical Convergence: PnP-ISTA with MMSE denoisers is proven to converge to a stationary point of a global cost function. The sequence of objective function values is monotonically decreasing.
  • Numerical Validation: Simulations on sparse signal recovery demonstrate that PnP-ISTA with both exact MMSE denoisers and approximate MMSE denoisers (learned DnCNNs) exhibit strong convergence and achieve comparable, high SNR performance.
  • Comparison to LASSO: The PnP-ISTA approaches with MMSE denoisers significantly outperform LASSO (ISTA with L1-norm prior) in terms of SNR.
  • Agreement of Denoisers: The excellent agreement between the exact and learned MMSE denoisers in terms of performance is encouraging, suggesting the practical utility of deep learning for approximating MMSE priors.

Limitations and Future Work

  • ISTA's Gaussian Noise Assumption: The paper acknowledges that the underlying assumption of ISTA (Gaussian noise at each iteration) is not strictly met when using MMSE denoisers. This might explain why PnP-ISTA is suboptimal compared to algorithms like GAMP, which are designed for random measurement matrices. Future work could explore more accurate error models for PnP iterations.
  • Suboptimality vs. GAMP: While PnP-ISTA with MMSE denoisers performs well, it's not as optimal as GAMP for sparse recovery with random matrices. This suggests a need for further research into PnP frameworks that better account for the statistical properties of the measurement process.
  • Generalization to Other PnP Algorithms: The current analysis focuses on PnP-ISTA. Extending these theoretical guarantees to other PnP algorithms (e.g., PnP-ADMM) would be valuable.
  • Characterizing Errors: The authors hypothesize that similar phenomena to those observed in sparse recovery might occur in imaging inverse problems, suggesting the need for refined statistical models to characterize errors within PnP algorithms.

Summary updated

Abstract

Plug-and-play priors (PnP) is a methodology for regularized image reconstruction that specifies the prior through an image denoiser. While PnP algorithms are well understood for denoisers performing maximum a posteriori probability (MAP) estimation, they have not been analyzed for the minimum mean squared error (MMSE) denoisers. This letter addresses this gap by establishing the first theoretical convergence result for the iterative shrinkage/thresholding algorithm (ISTA) variant of PnP for MMSE denoisers. We show that the iterates produced by PnP-ISTA with an MMSE denoiser converge to a stationary point of some global cost function. We validate our analysis on sparse signal recovery in compressive sensing by comparing two types of denoisers, namely the exact MMSE denoiser and the approximate MMSE denoiser obtained by training a deep neural net.
BibTeX
@article{xu2020provable,
  author = {Xiaojian Xu and Yu Sun and Jiaming Liu and Brendt Wohlberg and Ulugbek S. Kamilov},
  title = {{Provable Convergence of Plug-and-Play Priors With MMSE Denoisers}},
  journal = {IEEE Signal Processing Letters},
  year = {2020},
  url = {https://openreview.net/forum?id=8JhFxVR8OM},
}

Loading related content...

Figure