Hybrid Local-Global Transformer for Image Dehazing

Dong Zhao, Jia Li, Hongyu Li, Long XuarXiv 2021 · arXiv

Introduction

Image dehazing remains a challenging task in computer vision due to the complex nature of light scattering. Traditional methods often rely on the Atmospheric Scattering Model (ASM), which simplifies real-world physics and frequently fails in non-homogeneous or nighttime conditions. Conversely, modern deep learning approaches often act as "black boxes," lacking interpretability and struggling with long-range dependencies due to the local nature of standard convolutions.

This paper introduces the Complementary Feature Enhanced Network (CFEN), a framework designed to bridge the gap between physical interpretability and deep learning power. By decomposing images into intrinsic components and utilizing a novel Transformer architecture, CFEN achieves state-of-the-art performance across diverse hazy environments.

The Complementary Feature Enhanced Framework

The core of the proposed method is a multi-task learning strategy. Instead of a single-path mapping from hazy to clean images, the network employs a shared encoder followed by three specialized decoders.

Comparison of dehazing frameworks: (a) Physical-based, (b) Fully Learning-based, and (c) the proposed Complementary Feature Enhanced Framework.

As shown in Figure 1, the CFEN architecture (c) integrates subtasks for intrinsic image decomposition. Two decoders are dedicated to estimating reflectance and shading, which provide complementary information to the primary dehazing decoder. To manage these features, a Complementary Features Selection Module (CFSM) uses channel-wise attention to dynamically boost relevant information while suppressing noise.

Furthermore, the network incorporates the Hybrid Local-Global Vision Transformer (HyLoG-ViT). This block addresses the computational inefficiency of standard Transformers by splitting the workload into a local path for fine-grained details and a downsampled global path for long-range dependencies, ensuring a comprehensive receptive field without quadratic complexity.

Leveraging Intrinsic Image Decomposition

A key innovation of this work is using intrinsic image decomposition as a prior for dehazing. Haze typically degrades images in two distinct ways: color distortion and loss of texture. These map directly to the reflectance and shading components of an image.

Intrinsic image decomposition of a hazy image ($I_H$) into its reflectance ($I_{HR}$) and shading ($I_{HS}$) components.

By forcing the network to learn these components (Figure 4), the model gains a more structured understanding of the scene. Reflectance captures the underlying color information, while shading captures the structural and textural details. This explicit separation allows the final dehazing decoder to reconstruct the scene with higher fidelity, particularly in challenging areas like sky regions or distant objects.

Experimental Results

The CFEN model was rigorously evaluated on several benchmarks, including RESIDE (homogeneous haze), NH-HAZE (non-homogeneous haze), and NHR (nighttime haze).

Key quantitative highlights include:

  • O-HAZE: Achieved a PSNR of 29.87 dB, significantly outperforming previous leaders like EPDN (17.17 dB).
  • Nighttime Dehazing: On the NHR dataset, the model reached 22.94 dB PSNR, surpassing specialized models like OSFD.

Qualitative results demonstrate that CFEN effectively avoids over-saturation and preserves fine textures that are often lost or blurred by competing CNN-based architectures.

Conclusion

The Hybrid Local-Global Transformer framework demonstrates that combining global modeling with interpretable subtasks significantly improves image dehazing. While the multi-decoder approach introduces some computational overhead compared to simple CNNs, the gains in accuracy and robustness—especially in nighttime and non-homogeneous scenarios—are substantial. Future work may focus on optimizing the architecture for real-time deployment and extending the complementary feature framework to other restoration tasks like deraining or low-light enhancement.

Summary updated

Abstract

Conventional CNNs-based dehazing models suffer from two essential issues: the dehazing framework (limited in interpretability) and the convolution layers (content-independent and ineffective to learn long-range dependency information). In this paper, firstly, we propose a new complementary feature enhanced framework, in which the complementary features are learned by several complementary subtasks and then together serve to boost the performance of the primary task. One of the prominent advantages of the new framework is that the purposively chosen complementary tasks can focus on learning weakly dependent complementary features, avoiding repetitive and ineffective learning of the networks. We design a new dehazing network based on such a framework. Specifically, we select the intrinsic image decomposition as the complementary tasks, where the reflectance and shading prediction subtasks are used to extract the color-wise and texture-wise complementary features. To effectively aggregate these complementary features, we propose a complementary features selection module (CFSM) to select the more useful features for image dehazing. Furthermore, we introduce a new version of vision transformer block, named Hybrid Local-Global Vision Transformer (HyLoG-ViT), and incorporate it within our dehazing networks. The HyLoG-ViT block consists of the local and the global vision transformer paths used to capture local and global dependencies. As a result, the HyLoG-ViT introduces locality in the networks and captures the global and long-range dependencies. Extensive experiments on homogeneous, non-homogeneous, and nighttime dehazing tasks reveal that the proposed dehazing network can achieve comparable or even better performance than CNNs-based dehazing models.
BibTeX
@misc{zhao2021hybrid,
  author = {Dong Zhao and Jia Li and Hongyu Li and Long Xu},
  title = {{Hybrid Local-Global Transformer for Image Dehazing}},
  year = {2021},
  eprint = {2109.07100},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.07100},
}
Figure