Deep Multi-Model Fusion for Single-Image Dehazing

Zijun Deng, Lei Zhu, Xiaowei Hu, Chi-Wing Fu, Xuemiao Xu, Qing Zhang, Jing Qin, Pheng-Ann Heng

Abstract

Single-image dehazing is a notoriously difficult "ill-posed" problem where a single input must be used to estimate both the scene's depth and its original colors. This paper introduces DM$^2$F-Net, a deep learning architecture that achieves state-of-the-art results by fusing the traditional physical atmospheric scattering model with four empirical "layer separation" models, all guided by a clever multi-level attention mechanism.

Introduction

Capturing clear images in hazy or foggy conditions is a significant challenge for computer vision systems. Light absorption and scattering by atmospheric particles result in low contrast, faded colors, and a loss of fine detail. Mathematically, this phenomenon is typically modeled using the Atmospheric Scattering (AS) model, which represents a hazy image as a combination of clean scene radiance, a transmission map (indicating how much light reaches the sensor), and global atmospheric light.

However, most existing deep learning methods rely solely on the AS model. Since the AS model is a simplified approximation of complex real-world physics, these methods often suffer from "over-dehazing" (creating dark, unnatural artifacts) or "under-dehazing" (leaving a lingering film of fog). The authors of this paper argue that a single physical model is insufficient to capture the full complexity of real-world haze.

Methodology: The DM$^2$F-Net Architecture

The proposed Deep Multi-Model Fusion Network (DM$^2$F-Net) addresses these limitations through three primary innovations:

  1. Attentional Feature Integration Module (AFIM): Standard Convolutional Neural Networks (CNNs) often lose fine spatial detail in deeper layers while gaining semantic context. The AFIM uses an attention mechanism to intelligently merge features from both shallow and deep layers, ensuring the network preserves both high-level semantic information and fine-grained spatial details.
  2. Multi-Model Branching: Rather than relying on a single model, the network predicts five different versions of the dehazed image. One branch ($J_0$) follows the traditional AS model, while four additional branches ($J_1$ to $J_4$) utilize empirical Layer Separation models based on different mathematical operations (multiplication, addition, exponentiation, and logarithmic). These serve as empirical backups to the physical model.
  3. Attentional Fusion: The network learns five "weighting maps" ($W_0$ to $W_4$) to fuse these five results. This allows the system to dynamically prioritize the best-performing model for specific regions of an image. For instance, if the AS model struggles with high-contrast areas where an empirical model succeeds, the attention map will assign higher weight to the latter for those specific pixels.
The DM$^2$F-Net architecture, showing feature extraction, the AFIM, multi-model branching, and the final attentional fusion stage.

Experimental Evaluation

The model was trained and evaluated on a diverse set of synthetic and real-world benchmarks to ensure robustness:

  • O-HAZE: A challenging dataset of real-world outdoor scenes with professional-grade artificial haze.
  • HAZERD: Outdoor images with realistic synthetic haze.
  • SOTS (from RESIDE): A standard large-scale synthetic dehazing benchmark.
  • TestA-DCPDN: A synthetic dataset specifically for comparing against leading architectures.

Results and Discussion

DM$^2$F-Net significantly outperformed previous state-of-the-art methods across both quantitative and qualitative metrics. On the real-world O-HAZE dataset, it achieved a PSNR of 25.188, a substantial improvement over the previous best (DCPDN) at 22.777.

Qualitative comparison of dehazing results. (a) Input hazy image, (b) DM$^2$F-Net result, (c) and (d) results from competing methods.

Visual comparisons demonstrate that DM$^2$F-Net produces images with more realistic colors and fewer artifacts, such as halos or dark distortions, compared to older methods like AOD-Net or GFN. The model is particularly effective at restoring visibility in extremely thick haze where other models typically fail.

A key insight from the study is the synergy between physical and empirical models. By allowing the network to explore purely mathematical formulations alongside physics-based ones, the system compensates for the AS model's inherent blind spots. The learned weight maps act as a smart selector, often assigning different models to different parts of the image, such as using one model for the sky and another for complex foreground textures.

Conclusion and Future Work

While DM$^2$F-Net sets a new benchmark for daytime dehazing, the authors note a primary limitation: Nighttime Dehazing. The model currently struggles with the complex, multi-colored, and point-source lighting (e.g., streetlamps) found in night scenes, as the training data primarily consists of daytime images. Future research will focus on incorporating more diverse datasets, including nighttime and low-light hazy scenes, to enhance the model's utility for 24/7 applications like autonomous driving.

Summary updated

Abstract

This paper presents a deep multi-model fusion network to attentively integrate multiple models to separate layers and boost the performance in single-image dehazing. To do so, we first formulate the attentional feature integration module to maximize the integration of the convolutional neural network (CNN) features at different CNN layers and generate the attentional multi-level integrated features (AMLIF). Then, from the AMLIF, we further predict a haze-free result for an atmospheric scattering model, as well as for four haze-layer separation models, and then fuse the results together to produce the final haze-free image. To evaluate the effectiveness of our method, we compare our network with several state-of-the-art methods on two widely-used dehazing benchmark datasets, as well as on two sets of real-world hazy images. Experimental results demonstrate clear quantitative and qualitative improvements of our method over the state-of-the-arts.
BibTeX
@inproceedings{deng2019deep,
  author = {Zijun Deng and Lei Zhu and Xiaowei Hu and Chi-Wing Fu and Xuemiao Xu and Qing Zhang and Jing Qin and Pheng-Ann Heng},
  title = {{Deep Multi-Model Fusion for Single-Image Dehazing}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00254},
}

Loading related content...

Figure