Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization

Samuel Hurault, Arthur Leclaire, Nicolas Papadakis

Abstract

Plug-and-Play (PnP) methods solve ill-posed inverse problems through iterative proximal algorithms by replacing a proximal operator with a denoising operation. When applied with deep neural network denoisers, these methods have shown state-of-the-art visual performance for image restoration. However, their theoretical convergence analysis is still incomplete. Most existing results consider nonexpansive denoisers, which is unrealistic, or limit analysis to strongly convex data-fidelity terms. Recently, it was proposed to train the denoiser as a gradient descent step on a functional parameterized by a deep neural network. In this paper, we show that this gradient denoiser can actually correspond to the proximal operator of another scalar function. Given this result, we exploit the convergence theory of proximal algorithms in the nonconvex setting to obtain convergence results for PnP-PGD and PnP-ADMM. When built on top of a smooth gradient denoiser, we show that these methods target stationary points of an explicit functional, confirmed by numerical experiments on deblurring, super-resolution, and inpainting.

Introduction

Image restoration—recovering a clean image from a blurry, noisy, or downsampled version—is a fundamental challenge in computer vision. Traditionally, this is framed as an optimization problem: finding an image that fits the observed data while adhering to a "natural image" prior. Plug-and-Play (PnP) methods have revolutionized this field by replacing the mathematical prior with a pre-trained deep learning denoiser. While PnP methods achieve state-of-the-art results, they often lack theoretical stability; because deep denoisers are typically "black boxes," the iterative algorithms may oscillate or diverge, and the underlying objective function being minimized remains unknown.

The Challenge of Convergence

Previous attempts to guarantee PnP convergence required forcing the denoiser to be "nonexpansive" (a strict mathematical constraint where the denoiser does not increase the distance between points). Unfortunately, enforcing nonexpansivity often significantly degrades the denoiser's restoration performance. This paper addresses the gap by designing a denoiser that is mathematically well-behaved without sacrificing its power.

Designing a Proximal Denoiser

The authors introduce a Proximal Gradient Step Denoiser. The key innovation lies in the architecture and training constraints:

  1. Gradient Step Architecture: The denoiser is parameterized as $D_\sigma = \text{Id} - \nabla g_\sigma$, where $g_\sigma$ is a scalar function output by a neural network.
  2. The Proximal Connection: The authors prove that if the "residual" of this denoiser (the noise it removes) is contractive (with a Lipschitz constant $L < 1$), the denoiser is mathematically equivalent to a proximal operator of some function $\phi_\sigma$.
  3. Spectral Constraints: To ensure the $L < 1$ condition, they fine-tune a DRUNet architecture using a penalty on the spectral norm of the Hessian of $g_\sigma$. They also utilize Softplus activations to ensure the network is twice-differentiable ($C^2$).

Convergence in the Nonconvex Regime

Unlike classical theory that requires the regularizer $\phi_\sigma$ to be convex, this work leverages the Kurdyka-Lojasiewicz (KL) property. This allows the authors to prove that PnP-PGD (Proximal Gradient Descent), PnP-ADMM (Alternating Direction Method of Multipliers), and PnP-DRS (Douglas-Rachford Splitting) converge to stationary points of an explicit cost function, even when the underlying regularizer is nonconvex. This is a more natural fit for the complex manifold of natural images.

Experimental Results

The proposed methods were evaluated on standard benchmarks like CBSD68. In deblurring tasks, the Prox-PnP-DRS method achieved a PSNR of 28.07 dB, outperforming standard methods like IRCNN and providing a stable alternative to state-of-the-art methods like DPIR, which lack convergence guarantees.

Figure 1: Deblurring results and convergence analysis for the 'starfish' and sea anemone images. The plots (i-l) demonstrate the stable minimization of the objective function and the convergence of iterates over 500 iterations.

As shown in Figure 1, the proposed Prox-PnP variants not only produce high-quality visual restorations but also exhibit monotonic decrease in the objective function $F_{\lambda, \sigma}(x_k)$. This stability is a critical advantage over traditional PnP methods, which can diverge if run for too many iterations. In super-resolution tasks ($s=3$), the method achieved 25.29 dB, significantly outperforming IRCNN's 24.72 dB.

Conclusion

By bridging the gap between deep learning and proximal optimization, this work provides a rigorous framework for PnP methods. The realization that the residual mapping's contraction is the key constraint allows for high-performance denoisers to be used within a mathematically sound optimization loop. While challenges remain in verifying certain assumptions like the convexity of the denoiser's image space, this research marks a significant step toward reliable and interpretable deep image restoration.

Summary updated

Abstract

Plug-and-Play (PnP) methods solve ill-posed inverse problems through iterative proximal algorithms by replacing a proximal operator by a denoising operation. When applied with deep neural network denoisers, these methods have shown state-of-the-art visual performance for image restoration problems. However, their theoretical convergence analysis is still incomplete. Most of the existing convergence results consider nonexpansive denoisers, which is non-realistic, or limit their analysis to strongly convex data-fidelity terms in the inverse problem to solve. Recently, it was proposed to train the denoiser as a gradient descent step on a functional parameterized by a deep neural network. Using such a denoiser guarantees the convergence of the PnP version of the Half-Quadratic-Splitting (PnP-HQS) iterative algorithm. In this paper, we show that this gradient denoiser can actually correspond to the proximal operator of another scalar function. Given this new result, we exploit the convergence theory of proximal algorithms in the nonconvex setting to obtain convergence results for PnP-PGD (Proximal Gradient Descent) and PnP-ADMM (Alternating Direction Method of Multipliers). When built on top of a smooth gradient denoiser, we show that PnP-PGD and PnP-ADMM are convergent and target stationary points of an explicit functional. These convergence results are confirmed with numerical experiments on deblurring, super-resolution and inpainting.
BibTeX
@inproceedings{hurault2022proximal,
  author = {Samuel Hurault and Arthur Leclaire and Nicolas Papadakis},
  title = {{Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=wUMFyjWlS5},
}
Figure