Abstract
Single-image dehazing is a notoriously difficult "ill-posed" problem where a single input must be used to estimate both the scene's depth and its original colors. This paper introduces DM$^2$F-Net, a deep learning architecture that achieves state-of-the-art results by fusing the traditional physical atmospheric scattering model with four empirical "layer separation" models, all guided by a clever multi-level attention mechanism.
Introduction
Capturing clear images in hazy or foggy conditions is a significant challenge for computer vision systems. Light absorption and scattering by atmospheric particles result in low contrast, faded colors, and a loss of fine detail. Mathematically, this phenomenon is typically modeled using the Atmospheric Scattering (AS) model, which represents a hazy image as a combination of clean scene radiance, a transmission map (indicating how much light reaches the sensor), and global atmospheric light.
However, most existing deep learning methods rely solely on the AS model. Since the AS model is a simplified approximation of complex real-world physics, these methods often suffer from "over-dehazing" (creating dark, unnatural artifacts) or "under-dehazing" (leaving a lingering film of fog). The authors of this paper argue that a single physical model is insufficient to capture the full complexity of real-world haze.
Methodology: The DM$^2$F-Net Architecture
The proposed Deep Multi-Model Fusion Network (DM$^2$F-Net) addresses these limitations through three primary innovations:
- Attentional Feature Integration Module (AFIM): Standard Convolutional Neural Networks (CNNs) often lose fine spatial detail in deeper layers while gaining semantic context. The AFIM uses an attention mechanism to intelligently merge features from both shallow and deep layers, ensuring the network preserves both high-level semantic information and fine-grained spatial details.
- Multi-Model Branching: Rather than relying on a single model, the network predicts five different versions of the dehazed image. One branch ($J_0$) follows the traditional AS model, while four additional branches ($J_1$ to $J_4$) utilize empirical Layer Separation models based on different mathematical operations (multiplication, addition, exponentiation, and logarithmic). These serve as empirical backups to the physical model.
- Attentional Fusion: The network learns five "weighting maps" ($W_0$ to $W_4$) to fuse these five results. This allows the system to dynamically prioritize the best-performing model for specific regions of an image. For instance, if the AS model struggles with high-contrast areas where an empirical model succeeds, the attention map will assign higher weight to the latter for those specific pixels.
Experimental Evaluation
The model was trained and evaluated on a diverse set of synthetic and real-world benchmarks to ensure robustness:
- O-HAZE: A challenging dataset of real-world outdoor scenes with professional-grade artificial haze.
- HAZERD: Outdoor images with realistic synthetic haze.
- SOTS (from RESIDE): A standard large-scale synthetic dehazing benchmark.
- TestA-DCPDN: A synthetic dataset specifically for comparing against leading architectures.
Results and Discussion
DM$^2$F-Net significantly outperformed previous state-of-the-art methods across both quantitative and qualitative metrics. On the real-world O-HAZE dataset, it achieved a PSNR of 25.188, a substantial improvement over the previous best (DCPDN) at 22.777.
Visual comparisons demonstrate that DM$^2$F-Net produces images with more realistic colors and fewer artifacts, such as halos or dark distortions, compared to older methods like AOD-Net or GFN. The model is particularly effective at restoring visibility in extremely thick haze where other models typically fail.
A key insight from the study is the synergy between physical and empirical models. By allowing the network to explore purely mathematical formulations alongside physics-based ones, the system compensates for the AS model's inherent blind spots. The learned weight maps act as a smart selector, often assigning different models to different parts of the image, such as using one model for the sky and another for complex foreground textures.
Conclusion and Future Work
While DM$^2$F-Net sets a new benchmark for daytime dehazing, the authors note a primary limitation: Nighttime Dehazing. The model currently struggles with the complex, multi-colored, and point-source lighting (e.g., streetlamps) found in night scenes, as the training data primarily consists of daytime images. Future research will focus on incorporating more diverse datasets, including nighttime and low-light hazy scenes, to enhance the model's utility for 24/7 applications like autonomous driving.
Summary updated