Constrained Plug-and-Play Priors for Image Restoration

Alessandro Benfenati, Pasquale CascaranoJ. Imaging 2024 · DOI · OpenReview

TL;DR

This paper introduces Constrained Plug-and-Play (CPnP), a novel method for image restoration that reformulates the traditional Plug-and-Play (PnP) framework as a constrained optimization problem. By directly linking the regularization parameter to the estimated noise level, CPnP offers a more physically interpretable and robust approach, outperforming existing methods in stability and image quality.

What Problem Does This Paper Address?

Image restoration, a fundamental task in many imaging applications (medicine, microscopy, astronomy), often involves solving linear inverse problems. These problems are typically ill-posed, meaning a direct solution is unstable and sensitive to noise. Traditional model-based methods address this by minimizing a regularized objective function, where a regularization parameter ($\mu$) controls the trade-off between data fidelity and prior information. However, selecting this parameter is often a manual and challenging process, lacking clear physical interpretation.

The Plug-and-Play (PnP) framework emerged as a powerful alternative, allowing the use of pre-trained denoisers as implicit priors within model-based methods. This combines the flexibility of model-based approaches with the effectiveness of learned denoisers. Despite its success, the regularization strength in traditional PnP still lacks a direct physical meaning, leading to the same parameter tuning difficulties. This paper aims to bridge this gap by introducing a constrained formulation that directly links the regularization strength to the noise level in the observed data.

Methods

The core contribution of this paper is the Constrained Plug-and-Play (CPnP) method. It reformulates the standard PnP approach, which typically solves an unconstrained optimization problem of the form:

$x^* \in \text{argmin } l(x; A, b) + \mu\rho(x)$

into a constrained optimization problem:

$x^* \in \text{argmin } \rho(x) \quad \text{subject to } l(x; A, b) \leq \delta$

Here, $l(x; A, b)$ represents the data fidelity term (e.g., least squares for Gaussian noise), $\rho(x)$ is the regularization term (implicitly defined by a denoiser), and $\delta$ is a constraint that directly relates to the noise level in the measurements. The authors propose to set $\delta = \tau \sqrt{n}\sigma_\eta$, where $\sigma_\eta$ is the noise standard deviation and $\tau$ is a parameter in $[0,1]$ that allows for fine-tuning.

To solve this constrained problem efficiently, CPnP leverages the Alternating Direction Method of Multipliers (ADMM). The paper details how the ADMM framework is adapted to handle the constrained formulation, effectively replacing the proximity operator of the regularization term with a denoiser. This allows the regularization strength to be directly controlled by the constraint $\delta$, which is derived from the estimated noise level.

Datasets

The paper uses publicly available image datasets for evaluation:

  • Set5: A standard benchmark dataset for image restoration tasks.
  • Set24: Another dataset used for evaluating image restoration performance.
  • Kodak Lossless True Color Image Suite: Used for visual inspection in comparisons.

The authors simulate degraded data by applying a linear image formation model (blurring and additive Gaussian noise) to these datasets.

Key Insights and Ideas

  • Physical Interpretation of Regularization: The most significant insight is the direct link established between the regularization parameter and the noise level in the measurements. This moves away from arbitrary parameter tuning towards a physically meaningful constraint.
  • Robustness through Constraint: By enforcing a constraint related to the noise level, CPnP demonstrates improved stability and robustness compared to unconstrained PnP methods, even when hyperparameters are varied.
  • Denoiser as a Proxy for Prior: The paper reinforces the idea that advanced denoisers, particularly deep learning-based ones like DnCNN, can serve as highly effective implicit priors for image restoration.
  • ADMM as an Efficient Solver: The successful application of ADMM to solve the constrained CPnP problem highlights its versatility and efficiency for complex optimization tasks in image processing.
  • Automatic Parameter Selection: CPnP effectively automates the selection of the regularization parameter by tying it to the noise level, significantly reducing the burden of manual tuning.

Key Results

  • Quantitative Performance: CPnP consistently outperforms competing methods like PnP and RED in terms of PSNR and SSIM metrics across various datasets and degradation levels. For example, in Table 2, CPnP achieves higher PSNR and SSIM values compared to RED and PnP for both Set5 and Set24.
  • Qualitative Improvement: Visual comparisons (e.g., Figure 1, Figure 4, Figure 5) show that CPnP produces restorations with enhanced clarity, better preservation of fine details, and reduced noise compared to other methods.
  • Stability and Robustness: Experiments demonstrate that CPnP maintains high performance levels and stability even when varying ADMM penalty sequences and initializations (Figure 2).
  • Noise Level Estimation: The paper shows that the choice of the parameter $\tau$ in $\delta = \tau \sqrt{n}\sigma_\eta$ is crucial. While an ideal scenario might use $\tau=1$, a realistic scenario with estimated noise levels benefits from $\tau \approx 0.98$ due to potential overestimation by noise estimation algorithms.

Limitations and Future Work

  • Dependence on Noise Estimation: The performance of CPnP is inherently tied to the accuracy of the noise level estimation. While the paper uses a method from [43], inaccuracies in this estimation can affect the optimal choice of $\tau$.
  • Computational Cost: While ADMM is efficient, the inversion of matrices in the update steps can still be computationally intensive, although the paper notes that this can be mitigated by using Fourier transforms for certain operators.
  • Future Work: The paper suggests exploring more sophisticated noise estimation techniques and investigating the application of CPnP to other inverse problems beyond image restoration. Further research into the theoretical convergence guarantees for CPnP with various denoisers is also a promising direction.

Summary updated

Abstract

The Plug-and-Play framework has demonstrated that a denoiser can implicitly serve as the image prior for model-based methods for solving various inverse problems such as image restoration tasks. This characteristic enables the integration of the flexibility of model-based methods with the effectiveness of learning-based denoisers. However, the regularization strength induced by denoisers in the traditional Plug-and-Play framework lacks a physical interpretation, necessitating demanding parameter tuning. This paper addresses this issue by introducing the Constrained Plug-and-Play (CPnP) method, which reformulates the traditional PnP as a constrained optimization problem. In this formulation, the regularization parameter directly corresponds to the amount of noise in the measurements. The solution to the constrained problem is obtained through the design of an efficient method based on the Alternating Direction Method of Multipliers (ADMM). Our experiments demonstrate that CPnP outperforms competing methods in terms of stability and robustness while also achieving competitive performance for image quality.
BibTeX
@article{benfenati2024constrained,
  author = {Alessandro Benfenati and Pasquale Cascarano},
  title = {{Constrained Plug-and-Play Priors for Image Restoration}},
  journal = {J. Imaging},
  year = {2024},
  doi = {10.3390/jimaging10020050},
}
Figure