Blind Universal Bayesian Image Denoising with Gaussian Noise Level Learning

Majed El Helou, Sabine Süsstrunk

TL;DR

This paper introduces a novel deep learning approach for blind universal image denoising, meaning a single model can remove Gaussian noise from images without knowing the noise level beforehand. They derive a theoretically optimal "fusion denoising" method and adapt it into a practical network (BUIFD) that significantly improves denoising performance, especially on unseen noise levels, for both synthetic and real-world images.

What Problem Does This Paper Address?

Image denoising is a crucial step in many image processing tasks. Traditional methods often require knowing the exact noise level of an image, which is impractical in real-world scenarios. Furthermore, developing a single model that can handle a wide range of noise levels (universal) without needing to know the specific level at test time (blind) is a significant challenge. Existing deep learning models, while powerful, often lack interpretability and struggle to generalize to noise levels they weren't explicitly trained on. This paper aims to bridge this gap by developing a theoretically grounded, blind, and universal denoiser that performs well across various noise conditions.

Methods

The core of the paper lies in the concept of "fusion denoising." This approach is derived from Bayesian estimation theory, assuming a Gaussian prior for the image. It essentially fuses the noisy image with a learned image prior, weighted by the signal-to-noise ratio (SNR).

The authors propose two main network architectures:

  1. Fusion Net: This network is designed for synthetic data and directly implements the theoretically derived fusion denoising. It uses components similar to the DnCNN architecture but explicitly learns the SNR function and the image prior. An auxiliary loss function is introduced to encourage the network to disentangle these two components.
  2. Blind Universal Image Fusion Denoiser (BUIFD): This is the practical adaptation for real-world images where the exact prior is unknown. BUIFD modifies the fusion step by replacing the direct mathematical fusion with a product fusion followed by trainable convolution layers. This allows the network to learn a data-dependent fusion function.

Both networks leverage residual learning and are trained to minimize reconstruction loss, aiming to approach the optimal Bayesian estimator.

Datasets

  • Synthetic Data: Over 200,000 patches of size 40x40 pixels were generated for training, with pixel intensities drawn from a Gaussian distribution.
  • Testing Data (Synthetic): 256 images of size 256x256 pixels were used for synthetic testing.
  • Real-World Datasets:
    • BSD68: A standard dataset for grayscale image denoising.
  • CBSD68: The color image denoising counterpart to BSD68.
  • Set5, Set14, Sun_Hays80, Urban100, Manga109: Additional datasets used for extended benchmark comparisons.

Key Insights and Ideas

  • Theoretical Grounding for Deep Learning: The paper demonstrates how a theoretically derived optimal solution (fusion denoising) can inform the design of deep learning architectures, leading to more interpretable and generalizable models.
  • SNR Learning is Crucial for Generalization: Explicitly learning the signal-to-noise ratio (SNR) is key to enabling the network to adapt its fusion strategy to different noise levels, thus improving generalization to unseen conditions.
  • Disentangling Prior and Noise Information: By separating the learning of the image prior and the noise level (or SNR), the network can better capture the underlying image structure and noise characteristics.
  • Adaptability to Real-World Images: The BUIFD architecture shows that the core principles of fusion denoising can be successfully adapted to real images, even when the exact image prior is unknown, by using learned fusion layers.

Key Results

  • Synthetic Experiments: The Fusion Net significantly outperformed the DnCNN on unseen Gaussian noise levels, demonstrating superior generalization capabilities.
  • Real-World Grayscale Denoising (BSD68):
    • BUIFD improved PSNR by up to 0.7dB on noise levels seen during training.
  • Crucially, BUIFD achieved improvements of up to 2.82dB on noise levels not seen during training, highlighting its strong generalization.
  • It also outperformed the non-blind BM3D by an average of 0.25dB when trained on all noise levels.
  • Real-World Color Denoising (CBSD68): BUIFD improved state-of-the-art color image denoising performance by an average of 0.1dB across all noise levels, whether trained on them or not.
  • Qualitative Results: Visual examples show that BUIFD better removes noise and preserves details compared to DnCNN, especially at higher noise levels, mitigating blurring artifacts.

Limitations and Future Work

  • Gaussian Noise Assumption: The theoretical framework is based on an additive white Gaussian noise model. While this is a common assumption, real-world noise can be more complex. The paper acknowledges that the BUIFD network is adapted to handle real images, but the theoretical foundation is still rooted in Gaussian noise.
  • Interpretability of Real-World Priors: While the network learns a prior, the paper notes that it's challenging to definitively assess how well this learned prior aligns with the true expected mean of the unknown distribution in real images.
  • Future Work: The authors suggest that their methodology could motivate future work in analyzing deep network optimality on theoretical problems with known solutions and inspire the design of new networks based on Bayesian principles. They also highlight the potential for further exploration of disentangled representations for various computer vision tasks.

Summary updated

Abstract

Blind and universal image denoising consists of using a unique model that denoises images with any level of noise. It is especially practical as noise levels do not need to be known when the model is developed or at test time. We propose a theoretically-grounded blind and universal deep learning image denoiser for additive Gaussian noise removal. Our network is based on an optimal denoising solution, which we call fusion denoising. It is derived theoretically with a Gaussian image prior assumption. Synthetic experiments show our network's generalization strength to unseen additive noise levels. We also adapt the fusion denoising network architecture for image denoising on real images. Our approach improves real-world grayscale additive image denoising PSNR results for training noise levels and further on noise levels not seen during training. It also improves state-of-the-art color image denoising performance on every single noise level, by an average of 0.1dB, whether trained on or not.
BibTeX
@inproceedings{helou2019blind,
  author = {Majed El Helou and Sabine Süsstrunk},
  title = {{Blind Universal Bayesian Image Denoising with Gaussian Noise Level Learning}},
  booktitle = {Infoscience},
  year = {2019},
  doi = {10.1109/tip.2020.2976814},
}
Figure