FD-GAN: Generative Adversarial Networks with Fusion-Discriminator for Single Image Dehazing

Yu Dong, Yihao Liu, He Zhang, Shifeng Chen, Yu Qiao

TL;DR

This paper introduces FD-GAN, a fully end-to-end Generative Adversarial Network for single image dehazing that bypasses the need to estimate intermediate parameters like transmission maps. Its core innovation is a "Fusion-discriminator" that leverages frequency information (high and low pass) as additional priors, leading to more natural and artifact-free dehazed images.

What Problem Does This Paper Address?

Hazy images suffer from reduced contrast, color fidelity, and edge information, hindering various computer vision tasks. Traditional dehazing methods often rely on a two-step process: estimating intermediate parameters (medium transmission and atmospheric light) and then recovering the haze-free image. However, accurately estimating these parameters is challenging due to a lack of robust priors and constraints, often leading to artifacts, color distortion, and incomplete haze removal. Existing learning-based methods, while improving, still often follow this indirect approach.

Methods

The proposed method, FD-GAN, employs a fully end-to-end generative adversarial network (GAN). The generator is a densely connected encoder-decoder architecture designed to directly output a haze-free image from a hazy input, maximizing information flow between layers.

The key novelty lies in the Fusion-discriminator. Instead of just comparing the generated image to the ground truth, it incorporates frequency information as additional priors. An image is decomposed into low-frequency (LF) components (smooth regions, brightness, color) and high-frequency (HF) components (edges, textures, fine details). The Fusion-discriminator takes the generated image along with its LF and HF components, and compares them to the LF and HF components of the ground truth image. This allows the discriminator to better distinguish between real and fake images by leveraging these frequency-based characteristics, guiding the generator to produce more realistic and detailed results.

The overall loss function combines:

  • Pixel-wise loss (L1 loss): Measures direct pixel differences.
  • SSIM loss: Captures structural similarity.
  • Perceptual loss: Measures similarity in feature space using VGG16.
  • Adversarial loss: Drives the generator to produce realistic outputs.

Datasets

  • Training: The authors synthesized a new, large-scale training dataset based on the Microsoft COCO dataset. This dataset includes a diverse range of both indoor and outdoor hazy images, aiming to improve model generalization compared to datasets solely based on indoor images (like NYU).
  • Testing: The model was evaluated on standard public datasets:
  • Synthetic Objective Testing Set (SOTS) from RESIDE.
  • NTIRE'18 challenge dataset.
  • Real-world hazy images were also used to assess generalization.

Key Insights and Ideas

  • Frequency Information as a Powerful Prior: The paper demonstrates that explicitly incorporating frequency domain information (LF and HF components) into the discriminator significantly improves dehazing performance. This is because these components capture distinct visual characteristics that are heavily affected by haze.
  • End-to-End Dehazing is Viable and Effective: By designing a generator that directly outputs haze-free images and a sophisticated discriminator, the method bypasses the problematic intermediate parameter estimation step, leading to cleaner results.
  • Training Data is Crucial: The authors highlight that the performance of learning-based dehazing methods is highly sensitive to the training data. Their custom dataset, incorporating diverse indoor and outdoor scenes, leads to superior results.
  • Fusion-Discriminator Strengths: Experiments show that combining both LF and HF information in the Fusion-discriminator (Fusion-full) yields the best results, suggesting that both aspects are important for comprehensive haze removal and realism.

Key Results

  • State-of-the-Art Performance: FD-GAN achieved the highest PSNR and SSIM scores on both SOTS and NTIRE'18 synthetic datasets, outperforming several established methods.
  • Visually Superior Results: Qualitative comparisons show FD-GAN generating more natural, visually pleasing dehazed images with less color distortion and fewer artifacts compared to other methods. It effectively removes haze even in heavily hazy areas.
  • Improved Generalization: The model trained on the authors' synthesized dataset showed better performance and generalization on real-world hazy images.
  • Ablation Study Success: The Fusion-discriminator significantly improved upon baseline models (L1+SSIM loss) and standard GANs, demonstrating its effectiveness.

Limitations and Future Work

  • Limitations: While the paper demonstrates strong performance, the authors acknowledge that dehazing is an inherently ill-posed problem. The effectiveness of the Fusion-discriminator is shown to be dependent on the specific fusion strategy (HF, LF, or both).
  • Future Work: The paper suggests further exploration into different variants of the Fusion-discriminator and potentially other frequency-related features. The emphasis on training data quality also implies that creating even larger and more diverse datasets could lead to further improvements.

Summary updated

Abstract

Recently, convolutional neural networks (CNNs) have achieved great improvements in single image dehazing and attained much attention in research. Most existing learning-based dehazing methods are not fully end-to-end, which still follow the traditional dehazing procedure: first estimate the medium transmission and the atmospheric light, then recover the haze-free image based on the atmospheric scattering model. However, in practice, due to lack of priors and constraints, it is hard to precisely estimate these intermediate parameters. Inaccurate estimation further degrades the performance of dehazing, resulting in artifacts, color distortion and insufficient haze removal. To address this, we propose a fully end-to-end Generative Adversarial Networks with Fusion-discriminator (FD-GAN) for image dehazing. With the proposed Fusion-discriminator which takes frequency information as additional priors, our model can generator more natural and realistic dehazed images with less color distortion and fewer artifacts. Moreover, we synthesize a large-scale training dataset including various indoor and outdoor hazy images to boost the performance and we reveal that for learning-based dehazing methods, the performance is strictly influenced by the training data. Experiments have shown that our method reaches state-of-the-art performance on both public synthetic datasets and real-world images with more visually pleasing dehazed results.
BibTeX
@inproceedings{dong2020fd,
  author = {Yu Dong and Yihao Liu and He Zhang and Shifeng Chen and Yu Qiao},
  title = {{FD-GAN: Generative Adversarial Networks with Fusion-Discriminator for Single Image Dehazing}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6701},
}
Figure