Deep Gaussian Denoiser Epistemic Uncertainty and Decoupled Dual-Attention Fusion

Xiaoqi Ma, Xiaoyu Lin, Majed El Helou, Sabine Süsstrunk

Introduction

Deep learning models for image denoising have significantly advanced the state-of-the-art in removing Additive White Gaussian Noise (AWGN). However, performance has recently plateaued due to epistemic uncertainty—the inherent uncertainty a model has about data it hasn't fully mastered. While traditional model ensembles (averaging multiple networks) can mitigate this, they are computationally expensive and memory-intensive.

This paper introduces a lightweight "virtual ensemble" approach. Instead of training multiple models, the authors pass spatially and frequency-manipulated versions of a single image through a single frozen, pretrained network. These outputs are then intelligently combined using a novel decoupled dual-attention fusion mechanism to achieve superior results without the overhead of a standard ensemble.

The Virtual Ensemble: Spatial and Frequency Manipulations

To create a diversity of outputs from a single network, the authors generate 12 variations of the noisy input image:

  1. Spatial Manipulations (SM): Seven variations including rotations (90°, 180°, 270°) and vertical mirroring.
  2. Frequency Manipulations (FM): Five variations using Discrete Cosine Transform (DCT) masks to filter specific frequency bands.
Error distributions of randomly sampled pixels across spatial (SM) and frequency (FM) manipulations compared to the regular image.

A key insight of the research is that CNNs lack true spatial invariance; they produce slightly different errors when fed mirrored or rotated images. More importantly, the authors discovered that errors from frequency-manipulated images are highly decorrelated from spatial errors, making them exceptionally valuable for an ensemble. As shown in the error distribution analysis, these manipulations expose the model's uncertainty in ways that simple spatial flips cannot.

Decoupled Dual-Attention Fusion

Because the errors in these 13 outputs (12 manipulations + 1 original) are complex and not zero-centered, simple averaging actually degrades performance. To solve this, the authors developed a learned fusion network featuring two parallel attention mechanisms:

  • Spatial Attention: Learns pixel-specific weights, identifying which manipulation performed best in specific regions of the image.
  • Channel Attention: Evaluates the global quality of each denoised output.

By decoupling these mechanisms into parallel paths rather than sequential ones, the model avoids learning conflicts and redundant weights, allowing for a more effective merging of the "virtual ensemble" outputs.

Results and Performance

The proposed method was tested using standard baselines like DnCNN, MemNet, and RIDNet. The results demonstrate a consistent and significant boost in Peak Signal-to-Noise Ratio (PSNR) across various noise levels.

Visual comparison of denoising performance at σ=50. The proposed fusion method (d) significantly reduces artifacts compared to the DnCNN baseline (c).

At high noise levels (σ = 50), the DnCNN baseline achieves a PSNR of 24.96 dB. The proposed fusion method boosts this to 25.99 dB—a gain of over 1 dB without changing the underlying denoiser's weights. Qualitatively, the fusion method preserves sharp details (such as the penguin's beak in the figure above) that are often lost or blurred by baseline models.

Conclusion

By treating the base denoiser as a black box, this fusion method is entirely model-agnostic and can be applied to any future architecture. While it requires 13 forward passes per image, it offers a massive memory saving over traditional ensembles while providing a robust solution to epistemic uncertainty in image restoration tasks.

Summary updated

BibTeX
@inproceedings{ma2021deep,
  author = {Xiaoqi Ma and Xiaoyu Lin and Majed El Helou and Sabine Süsstrunk},
  title = {{Deep Gaussian Denoiser Epistemic Uncertainty and Decoupled Dual-Attention Fusion}},
  booktitle = {ICIP},
  year = {2021},
  url = {https://openreview.net/forum?id=K_7bvNR55V},
}
Figure