Abstract
In this paper, we propose a novel haze removal algorithm based on a new feature called the patch map. Conventional patch-based haze removal algorithms, such as the Dark Channel Prior (DCP), usually perform dehazing with a fixed patch size. However, this often leads to problems in recovered results, including oversaturation and color distortion. To address these issues, we designed an adaptive and automatic patch size selection model called the Patch Map Selection Network (PMS-Net). This convolutional neural network (CNN) generates a spatially varying patch map from the input image. Experimental results on both synthesized and real-world hazy images demonstrate that PMS-Net significantly outperforms state-of-the-art algorithms by effectively addressing the limitations of fixed patch sizes.
The "Goldilocks" Problem in Haze Removal
Haze removal is an inherently ill-posed problem in computer vision. The classic solution, the Dark Channel Prior (DCP), relies on the assumption that most non-sky patches in clear images contain pixels with very low intensity in at least one color channel. However, the effectiveness of DCP is highly sensitive to the chosen patch size.
Using a fixed patch size creates a difficult trade-off:
- Small Patches: These often fail in bright regions (like white walls or the sky), causing the algorithm to "over-dehaze," which results in dark, oversaturated, and color-distorted pixels.
- Large Patches: While better at handling bright regions, large patches tend to create "halo artifacts" around sharp edges and increase the computational burden.
PMS-Net was developed to find the "just right" patch size for every pixel in an image automatically.
Methods: The Patch Map Selection Network
The core innovation of this research is the Patch Map Selection Network (PMS-Net). Instead of attempting to output a clean image directly, the network predicts a "Patch Map"—a per-pixel guide that dictates the optimal patch size for the dehazing process.
1. Generating Ground Truth
To train the network, the authors created ground-truth patch maps by running traditional dehazing on hazy images using 120 different patch sizes. For every pixel, they identified which size produced a result closest to the known haze-free ground truth.
2. PMS-Net Architecture
The network utilizes an encoder-decoder structure with several specialized components designed for efficiency and accuracy:
- Multiscale U-module: Inspired by Wide-ResNet and Inception architectures, this module uses a pyramid of convolutional kernels (5x5, 3x3, and 1x1). This allows the network to capture features at multiple scales while reducing the total number of parameters by 30%.
- Multi-Deconv: An advanced up-sampling module that helps reconstruct the shape and structure of the patch map more accurately than standard deconvolution.
- Global Convolutional Network (GCN) & Boundary Refinement (BR): These modules ensure the predicted map respects the edges and boundaries within the image, preventing "bleeding" artifacts.
Key Insights
- Patch Size as a Feature: The optimal patch size is not a global hyperparameter but a spatially varying feature of the scene itself.
- Bright Regions Need Large Patches: In white, gray, or sky regions, the patch size must be larger to satisfy the dark channel prior. In darker, detailed regions, smaller patches are preferred to preserve clarity.
- Hybrid Physics-Deep Learning Approach: By using a CNN to predict the patch map and then applying a modified DCP, the system combines the interpretability of traditional physics-based models with the powerful feature extraction of deep learning.
Experimental Results
PMS-Net achieved top-tier results across all major metrics compared to nine other state-of-the-art algorithms, including DehazeNet and MSCNN.
- Quantitative Superiority: On the Test A dataset (NYU-depth2), PMS-Net achieved a PSNR of 19.21 (vs. DCP’s 16.55) and an SSIM of 0.868 (vs. DCP’s 0.823).
- Color Fidelity: The model showed a 34% improvement in the CIEDE2000 metric, proving it is significantly better at avoiding the "blue-shift" or darkening common in other methods.
- Visual Quality: Qualitative results on real-world images show that PMS-Net removes haze effectively without the "halo" effects or the "burnt" look seen in fixed-patch methods.
Conclusion and Future Work
PMS-Net demonstrates that adaptive patch selection is a robust solution for single-image dehazing. While the current model is optimized for daytime scenes with uniform atmospheric light, future work may explore extending this approach to night-time haze and further integrating the patch map prediction into a fully end-to-end differentiable reconstruction pipeline.
Summary updated