End-to-End Unpaired Image Denoising with Conditional Adversarial Networks

Zhiwei Hong, Xiaocheng Fan, Tao Jiang, Jianxing FengAAAI 2020 · DOI

Introduction

Deep learning has revolutionized image denoising, yet most state-of-the-art models rely on supervised learning with paired datasets—matching noisy images with perfectly clean ground truths. In critical domains like low-light photography or medical imaging (e.g., CT scans), obtaining such pairs is often impossible due to motion or safety concerns regarding radiation exposure. Furthermore, traditional unpaired methods frequently assume simplistic "white noise" models, failing to capture the complex, content-dependent noise distributions found in the real world.

This paper introduces UIDNet (Unpaired Image Denoising Network), an end-to-end framework designed to learn realistic noise distributions from unpaired data and perform high-quality restoration.

Methodology: The UIDNet Framework

UIDNet addresses the lack of paired data by learning to synthesize realistic noise that mimics the target distribution. The architecture consists of two primary components trained in an integrated, end-to-end fashion.

The overall architecture of the UIDNet framework, illustrating the noise generation (cGAN) and denoising pipeline.

1. Noise Learning via Conditional GAN

The first module is a Noise Learning Network based on a Conditional Generative Adversarial Network (cGAN). The generator (G) takes a clean image (x) and a random noise vector (z) to produce "learned noise" (ε), which is then added to the clean image to create "pseudo-noisy" data (x').

A key innovation is the "Sharpening" Trick: to prevent the Discriminator (D) from focusing on the underlying image content rather than the noise texture, the input to D is processed through a sharpening technique. By subtracting a local-mean filtered version of the image, the model isolates high-frequency noise patterns, forcing the GAN to learn the subtle "grain" of real-world noise.

2. Integrated Denoising

The generated pseudo-noisy images, now paired with their original clean sources, serve as training data for the Denoising Network. This network follows a residual learning approach, where the model learns to predict the noise component to be subtracted from the input. By training both the noise generator and the denoiser simultaneously, the system creates a feedback loop where the denoiser adapts to increasingly realistic synthetic noise.

Experimental Evaluation

The researchers validated UIDNet across three distinct scenarios: synthetic Gaussian noise, real-world photography (SIDD), and medical imaging (Mayo Clinic Low Dose CT Grand Challenge). The latter represents a high-stakes environment with complex, dose-dependent noise.

Example of a medical CT scan used to evaluate the model's performance on complex, content-dependent noise.

Results and Discussion

UIDNet demonstrated significant improvements over both traditional and specialized unpaired denoising methods.

  • Photographic Performance: On the SIDD benchmark, UIDNet achieved a PSNR of 32.48 dB and an SSIM of 0.897. Notably, it outperformed the specialized CBDNet in structural similarity, suggesting that learning the noise distribution is more effective than simple pixel-to-pixel mapping.
  • Medical Generalization: In the Mayo Clinic CT challenge, UIDNet showed superior generalization compared to supervised models like RED-CNN. While supervised models often struggle when test data deviates from the training set, UIDNet’s distribution-based learning allows it to adapt more robustly to varying noise characteristics.
  • Visual Fidelity: Qualitative analysis indicates that UIDNet avoids the "over-smoothing" common in many deep-learning denoisers. By accurately modeling the noise, the network preserves sharp edges and fine structural details that are often lost in translation.

Conclusion

UIDNet provides a robust solution for image restoration in the absence of paired training data. By leveraging a cGAN with a specialized sharpening discriminator, the framework successfully captures complex, content-dependent noise distributions. While GAN training remains computationally intensive, the ability to generate high-quality results in medical and photographic contexts without ground-truth pairs marks a significant step forward for practical image processing.

Summary updated

BibTeX
@inproceedings{hong2020end,
  author = {Zhiwei Hong and Xiaocheng Fan and Tao Jiang and Jianxing Feng},
  title = {{End-to-End Unpaired Image Denoising with Conditional Adversarial Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.5834},
}
Figure