Unfolded Proximal Neural Networks for Robust Image Gaussian Denoising

Hoang Trieu Vy Le, Audrey Repetti, Nelly Pustelnik

TL;DR

This paper introduces a unified framework for designing "unfolded proximal neural networks" (PNNs) for image denoising. By unrolling established optimization algorithms into neural network layers, these PNNs achieve competitive denoising performance with significantly fewer parameters than traditional deep learning models, demonstrating a promising synergy between optimization theory and deep learning.

What Problem Does This Paper Address?

Image denoising, a fundamental task in image processing, aims to recover a clean image from a noisy observation. A common approach is to formulate this as a Maximum A Posteriori (MAP) estimation problem, which is often solved using iterative proximal algorithms. While effective, these algorithms can be computationally intensive. Recently, researchers have explored "unfolding" these iterative algorithms into neural networks (PNNs) to leverage the power of deep learning for improved performance and efficiency. However, existing PNN approaches often lack a unified framework, making it difficult to compare different strategies and understand the underlying principles. This paper addresses this by proposing a unified framework for designing PNNs for Gaussian denoising, based on established proximal algorithms, and investigating different learning strategies to optimize their performance and robustness.

Methods

The core contribution is a unified framework for building PNNs by unrolling primal-dual proximal algorithms. Specifically, the authors adapt the dual Forward-Backward (DFB) and primal-dual Chambolle-Pock (CP) algorithms. They propose a primal-dual building block that can represent iterations of these algorithms, allowing for the incorporation of skip connections (akin to residual connections in CNNs) through inertial steps.

Two learning strategies are introduced:

  • Learned Normalized Operators (LNO): This strategy adheres to theoretical conditions for algorithm convergence, aiming for more robust PNNs.
  • Learned Flexible Operator (LFO): This strategy offers more flexibility in learning parameters, potentially leading to higher performance but possibly at the cost of robustness.

The framework is designed to be flexible, allowing different linear operators (like convolution filters) to be learned at each layer.

Datasets

  • Training:
    • BSDS500 dataset: Used for training with a fixed noise level.
  • ImageNet dataset: Used for training with a variable noise level.
  • Testing:
    • BSDS500 validation set: Used for evaluating denoising performance and robustness.

Key Insights and Ideas

  • Optimization Theory as a Design Principle for NNs: The paper strongly advocates for using theoretical insights from optimization algorithms to guide the design of neural network architectures. This leads to more interpretable and potentially more robust models.
  • Unfolding Proximal Algorithms is Efficient: By unrolling iterative optimization algorithms, PNNs can achieve state-of-the-art denoising performance with significantly fewer parameters and computational cost compared to traditional deep networks.
  • Inertial Steps Enable Skip Connections: The inclusion of inertial steps in the unfolded algorithms naturally leads to skip connections in the PNN architecture, which is a well-established technique for improving deep network performance.
  • LNO Strategy Enhances Robustness: The LNO learning strategy, which respects theoretical convergence conditions, leads to more robust PNNs, even if it doesn't always yield the absolute highest peak denoising performance. This highlights a crucial trade-off between performance and robustness.
  • Unified Framework Simplifies Design and Comparison: The proposed unified framework simplifies the design and comparison of different PNN architectures derived from proximal algorithms.

Key Results

  • Competitive Denoising Performance: The proposed PNNs achieve denoising performance comparable to state-of-the-art methods like DRUnet, and in some cases, outperform them, especially in terms of robustness.
  • Significant Parameter Reduction: The unfolded PNNs have drastically fewer learnable parameters (e.g., ~1000 times fewer than DRUnet), leading to lighter architectures and reduced computational cost.
  • Improved Robustness: The LNO strategy consistently leads to more robust PNNs, as evidenced by lower Lipschitz constants. The PNNs also show robustness when applied to non-Gaussian noise and in a plug-and-play deblurring task.
  • Effective in Deblurring: When integrated into a Plug-and-Play Forward-Backward (PnP-FB) algorithm for image deblurring, the proposed PNNs demonstrate competitive restoration performance and robustness.

Limitations and Future Work

  • Focus on Gaussian Denoising: The primary focus of this work is Gaussian denoising. While robustness to non-Gaussian noise and deblurring is explored, extending the framework to other noise distributions or imaging problems might require modifications to the data-fidelity term.
  • Extension to Different Imaging Problems: The authors suggest extending the study to PNNs designed for different imaging problems. This includes handling non-Gaussian noise distributions and exploring PNNs for general image restoration tasks where the data-fidelity term might not be strongly convex.
  • Exploration of Acceleration Strategies: For problems where the data-fidelity term is not strongly convex, different acceleration strategies might need to be explored.

Summary updated

Abstract

A common approach to solve inverse imaging problems relies on finding a maximum a posteriori (MAP) estimate of the original unknown image, by solving a minimization problem. In thiscontext, iterative proximal algorithms are widely used, enabling to handle non-smooth functions and linear operators. Recently, these algorithms have been paired with deep learning strategies, to further improve the estimate quality. In particular, proximal neural networks (PNNs) have been introduced, obtained by unrolling a proximal algorithm as for finding a MAP estimate, but over a fixed number of iterations, with learned linear operators and parameters. As PNNs are based on optimization theory, they are very flexible, and can be adapted to any image restoration task, as soon as a proximal algorithm can solve it. They further have much lighter architectures than traditional networks. In this article we propose a unified framework to build PNNs for the Gaussian denoising task, based on both the dual-FB and the primal-dual Chambolle-Pock algorithms. We further show that accelerated inertial versions of these algorithms enable skip connections in the associated NN layers. We propose different learning strategies for our PNN framework, and investigate their robustness (Lipschitz property) and denoising efficiency. Finally, we assess the robustness of our PNNs when plugged in a forward-backward algorithm for an image deblurring problem.
BibTeX
@article{le2024unfolded,
  author = {Hoang Trieu Vy Le and Audrey Repetti and Nelly Pustelnik},
  title = {{Unfolded Proximal Neural Networks for Robust Image Gaussian Denoising}},
  journal = {IEEE TIP},
  year = {2024},
  doi = {10.1109/tip.2024.3437219},
}
Figure