Indirect Domain Shift for Single Image Dehazing

Huan Liu, Jun ChenIEEE Access 2021 · OpenReview

TL;DR

This paper introduces an "Indirect Domain Shift" (IDS) framework for single image dehazing, arguing that directly mapping hazy to clear images is too difficult for standard CNNs. IDS guides the restoration process through multiple stages and diverse network branches, leading to significantly improved texture detail recovery and overall image quality compared to state-of-the-art methods.

What Problem Does This Paper Address?

Single image dehazing aims to remove atmospheric haze from a photograph to reveal the underlying scene. While deep convolutional neural networks (CNNs) have shown great success in many computer vision tasks, they often struggle with dehazing, particularly in faithfully recovering fine texture details. The authors propose that this difficulty stems from the large "domain gap" between hazy and clear images. Directly training a CNN to bridge this gap in one go is challenging because the loss function can become very small in the neighborhood of the target (clear image) domain, making it hard for the network to learn precise mappings for low-level vision tasks where small errors are noticeable. This paper tackles the problem of effectively and accurately bridging this large domain gap for high-quality image dehazing.

Methods

The core of the proposed solution is the Indirect Domain Shift (IDS) framework. Instead of a direct end-to-end mapping, IDS guides the dehazing process through a series of intermediate steps and diverse network architectures. Key components include:

  • Multi-scale Estimation: The network processes the image at different resolutions, starting with a coarse estimation and progressively refining it. This helps maintain global context while focusing on local details.
  • Multi-branch Diversity: The framework employs multiple parallel network branches, each with a different loss function (e.g., Mean Squared Error - MSE, Structural Similarity Index Measure - SSIM). This allows for diverse estimations of the clear image, providing different "confident neighborhoods" to narrow down the search space for the final output.
  • FusionNet: A final network component fuses the outputs from the diverse branches and the original hazy image. This stage uses both a content loss (weighted sum of MSE and perceptual loss) and an adversarial loss. The adversarial loss is crucial for generating perceptually pleasing results and recovering fine texture details by pushing the generated images towards the distribution of real clear images.

The paper also proposes two training schemes:

  • Hard IDS: Each module is trained independently, with gradients flowing only through that specific module. This is computationally less demanding and helps alleviate gradient vanishing issues.
  • Soft IDS: A global loss function combines losses from all modules, and the entire network is trained end-to-end via back-propagation. This approach allows modules to operate in the feature space, potentially leading to smoother transitions and better overall performance.

Datasets

The authors utilize the RESIDE-standard dataset for training and evaluation. This dataset includes:

  • Indoor Training Set (ITS): 13,990 synthetic hazy indoor images.
  • Outdoor Training Set (OTS): 296,695 synthetic hazy outdoor images.
  • Testing Set (SOTS): 500 synthetic hazy indoor/outdoor images with corresponding haze-free counterparts.

Additionally, they evaluate on real-world hazy image datasets:

  • O-Haze dataset
  • Dense-Haze dataset
  • Fattal et al. [13] dataset

Key Insights and Ideas

  • Domain Gap is the Bottleneck: The paper strongly argues that the primary reason for CNNs' limitations in dehazing is the vast difference between hazy and clear image domains, making direct mapping problematic.
  • Indirect Mapping is Key: By breaking down the complex mapping into smaller, guided steps (multi-scale, multi-branch), the network can learn more effectively.
  • Diversity Enhances Detail: Using multiple branches with different loss functions (MSE for pixel accuracy, SSIM for perceptual quality) allows the network to capture a richer set of image characteristics.
  • Adversarial Loss for Perceptual Quality: While PSNR might slightly decrease, the adversarial loss is essential for generating visually pleasing results with fine texture details, moving beyond simple pixel-wise accuracy.
  • Hard vs. Soft IDS: Both training schemes have merits; Hard IDS is simpler and addresses gradient issues, while Soft IDS offers potentially better performance through end-to-end feature learning.

Key Results

The proposed IDS framework, particularly the Soft IDS variant, significantly outperforms state-of-the-art dehazing methods across various datasets.

  • Quantitative Performance: On the SOTS indoor dataset, Soft IDS achieved a PSNR of 34.74 and an SSIM of 0.9873. When combined with RCANs (Residual Channel Attention Networks), the performance further improved.
  • Qualitative Improvements: The method successfully removes haze while preserving fine texture details, avoiding color distortion and over-enhancement issues common in prior methods. Figures in the paper demonstrate visually superior results, especially on challenging datasets like Dense-Haze.
  • Runtime: The IDS method is competitive in terms of runtime, ranking third among DNN-based methods on the SOTS dataset, with potential for further speedup through parallelization.

Limitations and Future Work

  • Dense-Haze Performance: The authors acknowledge that the visual performance on the Dense-Haze dataset is not completely satisfactory. They attribute this to the limited training data available for such challenging scenes.
  • Generalization with Limited Data: A key future direction is to enhance the IDS framework to achieve better generalization with limited training data.
  • Parallel Implementation: Further speed improvements are possible by parallelizing the multi-scale estimation across branches.
  • Exploring Alternative Architectures: The paper notes that RDNs are just one implementation choice, and using more powerful architectures like RCANs can further boost performance.

Summary updated

Abstract

Despite their remarkable expressibility, convolution neural networks (CNNs) still fall short of delivering satisfactory results on single image dehazing, especially in terms of faithful recovery of fine texture details. In this paper, we argue that the inadequacy of conventional CNN-based dehazing methods can be attributed to the fact that the domain of hazy images is too far away from that of clear images, rendering it difficult to train a CNN for learning direct domain shift through an end-to-end manner and recovering texture details simultaneously. To address this issue, we propose to add explicit constraints inside a deep CNN model to guide the restoration process. In contrast to direct learning, the proposed mechanism shifts and narrows the candidate region for the estimation output via multiple confident neighborhoods. Therefore, it is capable of consolidating the expressibility of different architectures, resulting in a more accurate indirect domain shift (IDS) from the hazy images to that of clear images. We also propose two different training schemes, including hard IDS and soft IDS, which further reveal the effectiveness of the proposed method. Our extensive experimental results indicate that the dehazing method based on this mechanism dramatically outperforms the state-of-the-arts.
BibTeX
@inproceedings{liu2021indirect,
  author = {Huan Liu and Jun Chen},
  title = {{Indirect Domain Shift for Single Image Dehazing}},
  booktitle = {IEEE Access},
  year = {2021},
  url = {https://openreview.net/forum?id=EWAJTyALdm},
}
Figure