Image denoising using group sparsity residual and external nonlocal self-similarity prior

Zhiyuan Zha, Xinggan Zhang, Qiong Wang, Yechao Bai, Lan Tang

TL;DR

This paper introduces a novel image denoising method that tackles the challenge of reconstructing clean images from noisy observations by leveraging both the "group sparsity residual" and an "external nonlocal self-similarity (NSS) prior." By framing denoising as minimizing this residual and using a learned NSS prior from natural images, the method achieves superior visual quality and detail preservation compared to existing state-of-the-art techniques.

What Problem Does This Paper Address?

Image denoising is a fundamental problem in computer vision, aiming to recover a clean image from a noisy version. While many advanced methods exploit the "nonlocal self-similarity" (NSS) prior – the idea that similar patches can be found elsewhere in an image – they often struggle to perfectly reconstruct the original clean image. This is because they primarily rely on the NSS characteristics of the noisy image, which can lead to artifacts and loss of fine details. The core challenge this paper addresses is how to more effectively leverage prior knowledge about natural images to overcome the limitations of solely using the noisy observation's NSS properties.

Methods

The paper proposes a novel approach centered around two key concepts:

  • Group Sparsity Residual: Instead of directly estimating the clean image's sparse representation, the authors introduce the "group sparsity residual." This residual quantifies the difference between the sparse codes of the noisy image patches and the true sparse codes of the clean image. The denoising problem is then reframed as minimizing this residual.
  • External NSS Prior: To estimate the true group sparse coefficients of the clean image, the method utilizes an "external NSS prior." This prior is learned from a dataset of natural images using a Gaussian Mixture Model (GMM). This learned prior captures the statistical regularities of natural image patches more effectively than relying solely on the noisy input.

The algorithm combines these ideas by:

  1. Extracting noisy image patches and forming groups of similar patches.
  2. Using the learned GMM-based NSS prior to estimate the sparse coefficients of the clean image for each group.
  3. Forming the "group sparsity residual" by comparing these estimated clean coefficients with the sparse coefficients derived from the noisy patches.
  4. Minimizing this residual to refine the sparse coefficients and reconstruct the denoised image.

Datasets

The paper uses the Kodak photoCD dataset for training the Gaussian Mixture Model (GMM) that captures the external NSS prior. For evaluation, they use 14 typical natural images (shown in Fig. 2) and test the denoising performance across various noise levels.

Key Insights and Ideas

  • The "Residual" is Key: Reframing denoising as minimizing the residual between noisy and clean sparse representations, rather than directly estimating the clean representation, is a crucial insight. This allows for a more targeted approach to error correction.
  • External Priors Matter: Relying on an NSS prior learned from a diverse set of natural images (external prior) proves more effective than solely analyzing the NSS properties of the noisy input image. This external knowledge acts as a strong guide for reconstruction.
  • GMM for Richer Priors: Using Gaussian Mixture Models (GMMs) to learn the NSS prior allows for capturing more complex statistical structures within natural image patch groups, leading to better estimations.
  • Preserving Details: The method's ability to preserve finer image details and reduce ringing artifacts suggests that its approach to modeling image structure is more robust than methods that tend to oversmooth.

Key Results

The proposed method demonstrates significant improvements over several state-of-the-art denoising algorithms, including BM3D, EPLL, NCSR, GID, LINE, PGPD, and aGMM.

  • Quantitative: The paper reports average PSNR improvements ranging from 0.14dB to 1.30dB across different noise levels compared to these methods. For instance, at a noise level of σ=40, the proposed method achieved a PSNR of 26.57dB, outperforming BM3D (25.94dB) and other methods.
  • Qualitative: Visually, the proposed method produces denoised images with finer details and significantly fewer undesirable ringing artifacts compared to competing methods, which often result in over-smoothing or noticeable artifacts.

Limitations and Future Work

  • Computational Complexity: While not explicitly detailed as a limitation, the use of GMM learning and iterative refinement suggests that the method might have higher computational demands compared to simpler denoising techniques.
  • Parameter Tuning: The paper mentions the need for adaptive determination of parameters like λ (balancing fidelity and regularization) and provides a table of parameter settings for different noise levels, implying that careful tuning is still required.
  • Future Work: The authors' work naturally opens avenues for exploring more sophisticated group sparsity models, advanced GMM learning techniques, and potentially extending this approach to other image restoration tasks like deblurring or super-resolution.

Summary updated

Abstract

Nonlocal image representation has been successfully used in many image-related inverse problems including denoising, deblurring and deblocking. However, due to a majority of reconstruction methods only exploit the nonlocal self-similarity (NSS) prior of the degraded observation image, it is very challenging to reconstruct the latent clean image directly from the noisy observation. In this paper we propose a novel model for image denoising via group sparsity residual and external NSS prior. To boost the performance of image denoising, the concept of group sparsity residual is proposed, and thus the problem of image denoising is transformed into one that reduces the group sparsity residual. Due to the fact that the groups contain a large amount of NSS information of natural images, we obtain a good estimation of the group sparse coefficients of the original image by the external NSS prior based on Gaussian Mixture model (GMM) learning and the group sparse coefficients of noisy image are used to approximate the estimation. Experimental results demonstrate that the proposed approach not only outperforms many state-of-the-art methods, but also delivers the best qualitative denoising results with finer details and less ringing artifacts.
BibTeX
@inproceedings{zha2017image,
  author = {Zhiyuan Zha and Xinggan Zhang and Qiong Wang and Yechao Bai and Lan Tang},
  title = {{Image denoising using group sparsity residual and external nonlocal self-similarity prior}},
  booktitle = {ICIP},
  year = {2017},
  url = {https://openreview.net/forum?id=P6WzXWuUFn},
}
Figure