PMS-Net: Robust Haze Removal Based on Patch Map for Single Images

Wei-Ting Chen, Jian-Jiun Ding, Sy-Yen KuoCVPR 2019 · OpenReview

Abstract

In this paper, we propose a novel haze removal algorithm based on a new feature called the patch map. Conventional patch-based haze removal algorithms, such as the Dark Channel Prior (DCP), usually perform dehazing with a fixed patch size. However, this often leads to problems in recovered results, including oversaturation and color distortion. To address these issues, we designed an adaptive and automatic patch size selection model called the Patch Map Selection Network (PMS-Net). This convolutional neural network (CNN) generates a spatially varying patch map from the input image. Experimental results on both synthesized and real-world hazy images demonstrate that PMS-Net significantly outperforms state-of-the-art algorithms by effectively addressing the limitations of fixed patch sizes.

The "Goldilocks" Problem in Haze Removal

Haze removal is an inherently ill-posed problem in computer vision. The classic solution, the Dark Channel Prior (DCP), relies on the assumption that most non-sky patches in clear images contain pixels with very low intensity in at least one color channel. However, the effectiveness of DCP is highly sensitive to the chosen patch size.

Using a fixed patch size creates a difficult trade-off:

  • Small Patches: These often fail in bright regions (like white walls or the sky), causing the algorithm to "over-dehaze," which results in dark, oversaturated, and color-distorted pixels.
  • Large Patches: While better at handling bright regions, large patches tend to create "halo artifacts" around sharp edges and increase the computational burden.

PMS-Net was developed to find the "just right" patch size for every pixel in an image automatically.

Methods: The Patch Map Selection Network

The core innovation of this research is the Patch Map Selection Network (PMS-Net). Instead of attempting to output a clean image directly, the network predicts a "Patch Map"—a per-pixel guide that dictates the optimal patch size for the dehazing process.

1. Generating Ground Truth

To train the network, the authors created ground-truth patch maps by running traditional dehazing on hazy images using 120 different patch sizes. For every pixel, they identified which size produced a result closest to the known haze-free ground truth.

2. PMS-Net Architecture

The network utilizes an encoder-decoder structure with several specialized components designed for efficiency and accuracy:

  • Multiscale U-module: Inspired by Wide-ResNet and Inception architectures, this module uses a pyramid of convolutional kernels (5x5, 3x3, and 1x1). This allows the network to capture features at multiple scales while reducing the total number of parameters by 30%.
  • Multi-Deconv: An advanced up-sampling module that helps reconstruct the shape and structure of the patch map more accurately than standard deconvolution.
  • Global Convolutional Network (GCN) & Boundary Refinement (BR): These modules ensure the predicted map respects the edges and boundaries within the image, preventing "bleeding" artifacts.
The predicted patch map (f) generated by PMS-Net, which adaptively selects patch sizes for each pixel to optimize dehazing.

Key Insights

  • Patch Size as a Feature: The optimal patch size is not a global hyperparameter but a spatially varying feature of the scene itself.
  • Bright Regions Need Large Patches: In white, gray, or sky regions, the patch size must be larger to satisfy the dark channel prior. In darker, detailed regions, smaller patches are preferred to preserve clarity.
  • Hybrid Physics-Deep Learning Approach: By using a CNN to predict the patch map and then applying a modified DCP, the system combines the interpretability of traditional physics-based models with the powerful feature extraction of deep learning.

Experimental Results

PMS-Net achieved top-tier results across all major metrics compared to nine other state-of-the-art algorithms, including DehazeNet and MSCNN.

  • Quantitative Superiority: On the Test A dataset (NYU-depth2), PMS-Net achieved a PSNR of 19.21 (vs. DCP’s 16.55) and an SSIM of 0.868 (vs. DCP’s 0.823).
  • Color Fidelity: The model showed a 34% improvement in the CIEDE2000 metric, proving it is significantly better at avoiding the "blue-shift" or darkening common in other methods.
  • Visual Quality: Qualitative results on real-world images show that PMS-Net removes haze effectively without the "halo" effects or the "burnt" look seen in fixed-patch methods.
The final dehazed result (d) produced by PMS-Net, demonstrating superior color fidelity and natural detail preservation compared to fixed-patch methods.

Conclusion and Future Work

PMS-Net demonstrates that adaptive patch selection is a robust solution for single-image dehazing. While the current model is optimized for daytime scenes with uniform atmospheric light, future work may explore extending this approach to night-time haze and further integrating the patch map prediction into a fully end-to-end differentiable reconstruction pipeline.

Summary updated

Abstract

In this paper, we proposed a novel haze removal algorithm based on a new feature called the patch map. Conventional patch-based haze removal algorithms (e.g. the Dark Channel prior) usually performs dehazing with a fixed patch size. However, it may produce several problems in recovered results such as oversaturation and color distortion. Therefore, in this paper, we designed an adaptive and automatic patch size selection model called the Patch Map Selection Network (PMS-Net) to select the patch size corresponding to each pixel. This network is designed based on the convolutional neural network (CNN), which can generate the patch map from the image to image. Experimental results on both synthesized and real-world hazy images show that, with the combination of the proposed PMS-Net, the performance in haze removal is much better than that of other state-of-the-art algorithms and we can address the problems caused by the fixed patch size.
BibTeX
@inproceedings{chen2019pms,
  author = {Wei-Ting Chen and Jian-Jiun Ding and Sy-Yen Kuo},
  title = {{PMS-Net: Robust Haze Removal Based on Patch Map for Single Images}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=NIKblV44Lov},
}
Figure