TL;DR
This paper introduces an "Indirect Domain Shift" (IDS) framework for single image dehazing, arguing that directly mapping hazy to clear images is too difficult for standard CNNs. IDS guides the restoration process through multiple stages and diverse network branches, leading to significantly improved texture detail recovery and overall image quality compared to state-of-the-art methods.
What Problem Does This Paper Address?
Single image dehazing aims to remove atmospheric haze from a photograph to reveal the underlying scene. While deep convolutional neural networks (CNNs) have shown great success in many computer vision tasks, they often struggle with dehazing, particularly in faithfully recovering fine texture details. The authors propose that this difficulty stems from the large "domain gap" between hazy and clear images. Directly training a CNN to bridge this gap in one go is challenging because the loss function can become very small in the neighborhood of the target (clear image) domain, making it hard for the network to learn precise mappings for low-level vision tasks where small errors are noticeable. This paper tackles the problem of effectively and accurately bridging this large domain gap for high-quality image dehazing.
Methods
The core of the proposed solution is the Indirect Domain Shift (IDS) framework. Instead of a direct end-to-end mapping, IDS guides the dehazing process through a series of intermediate steps and diverse network architectures. Key components include:
- Multi-scale Estimation: The network processes the image at different resolutions, starting with a coarse estimation and progressively refining it. This helps maintain global context while focusing on local details.
- Multi-branch Diversity: The framework employs multiple parallel network branches, each with a different loss function (e.g., Mean Squared Error - MSE, Structural Similarity Index Measure - SSIM). This allows for diverse estimations of the clear image, providing different "confident neighborhoods" to narrow down the search space for the final output.
- FusionNet: A final network component fuses the outputs from the diverse branches and the original hazy image. This stage uses both a content loss (weighted sum of MSE and perceptual loss) and an adversarial loss. The adversarial loss is crucial for generating perceptually pleasing results and recovering fine texture details by pushing the generated images towards the distribution of real clear images.
The paper also proposes two training schemes:
- Hard IDS: Each module is trained independently, with gradients flowing only through that specific module. This is computationally less demanding and helps alleviate gradient vanishing issues.
- Soft IDS: A global loss function combines losses from all modules, and the entire network is trained end-to-end via back-propagation. This approach allows modules to operate in the feature space, potentially leading to smoother transitions and better overall performance.
Datasets
The authors utilize the RESIDE-standard dataset for training and evaluation. This dataset includes:
- Indoor Training Set (ITS): 13,990 synthetic hazy indoor images.
- Outdoor Training Set (OTS): 296,695 synthetic hazy outdoor images.
- Testing Set (SOTS): 500 synthetic hazy indoor/outdoor images with corresponding haze-free counterparts.
Additionally, they evaluate on real-world hazy image datasets:
- O-Haze dataset
- Dense-Haze dataset
- Fattal et al. [13] dataset
Key Insights and Ideas
- Domain Gap is the Bottleneck: The paper strongly argues that the primary reason for CNNs' limitations in dehazing is the vast difference between hazy and clear image domains, making direct mapping problematic.
- Indirect Mapping is Key: By breaking down the complex mapping into smaller, guided steps (multi-scale, multi-branch), the network can learn more effectively.
- Diversity Enhances Detail: Using multiple branches with different loss functions (MSE for pixel accuracy, SSIM for perceptual quality) allows the network to capture a richer set of image characteristics.
- Adversarial Loss for Perceptual Quality: While PSNR might slightly decrease, the adversarial loss is essential for generating visually pleasing results with fine texture details, moving beyond simple pixel-wise accuracy.
- Hard vs. Soft IDS: Both training schemes have merits; Hard IDS is simpler and addresses gradient issues, while Soft IDS offers potentially better performance through end-to-end feature learning.
Key Results
The proposed IDS framework, particularly the Soft IDS variant, significantly outperforms state-of-the-art dehazing methods across various datasets.
- Quantitative Performance: On the SOTS indoor dataset, Soft IDS achieved a PSNR of 34.74 and an SSIM of 0.9873. When combined with RCANs (Residual Channel Attention Networks), the performance further improved.
- Qualitative Improvements: The method successfully removes haze while preserving fine texture details, avoiding color distortion and over-enhancement issues common in prior methods. Figures in the paper demonstrate visually superior results, especially on challenging datasets like Dense-Haze.
- Runtime: The IDS method is competitive in terms of runtime, ranking third among DNN-based methods on the SOTS dataset, with potential for further speedup through parallelization.
Limitations and Future Work
- Dense-Haze Performance: The authors acknowledge that the visual performance on the Dense-Haze dataset is not completely satisfactory. They attribute this to the limited training data available for such challenging scenes.
- Generalization with Limited Data: A key future direction is to enhance the IDS framework to achieve better generalization with limited training data.
- Parallel Implementation: Further speed improvements are possible by parallelizing the multi-scale estimation across branches.
- Exploring Alternative Architectures: The paper notes that RDNs are just one implementation choice, and using more powerful architectures like RCANs can further boost performance.
Summary updated