Image Dehazing Transformer with Transmission-Aware 3D Position Embedding

Chunle Guo, Qixin Yan, Saeed Anwar, Runmin Cong, Wenqi Ren, Chongyi Li

Introduction

Single-image dehazing remains a fundamental challenge in computer vision. The goal is to recover a clear, high-contrast scene from an image degraded by atmospheric scattering. While traditional Convolutional Neural Networks (CNNs) excel at capturing local textures, they often fail to model long-range dependencies necessary for understanding global haze distribution. Conversely, standard Transformers provide excellent global context but frequently produce coarse results that lose fine structural details.

Furthermore, standard Transformer architectures rely on 2D position embeddings (x, y coordinates). In dehazing, however, the degradation is not just a function of spatial location but also of scene depth and haze density. To address these limitations, the authors propose DeHamer (DeHazing transforMER), a hybrid architecture that integrates a novel Transmission-Aware 3D Position Embedding.

The DeHamer Architecture

DeHamer utilizes a dual-path approach to balance global and local feature extraction. The model consists of a Transformer module (based on the Swin Transformer) to capture hierarchical global features and a parallel CNN encoder to preserve local details.

The DeHamer architecture combines a Transformer module for global context with a CNN encoder for local details, linked by a feature modulation module.

The "glue" of this architecture is the Feature Modulation Module. Unlike standard fusion techniques like concatenation or addition, which can introduce artifacts, DeHamer uses Transformer-derived features to modulate the CNN features. By learning scaling and shifting factors, the global context effectively "tunes" the local representations, ensuring a more coherent reconstruction.

Transmission-Aware 3D Position Embedding (TA-3DPE)

The core innovation of DeHamer is the Transmission-Aware 3D Position Embedding (TA-3DPE). Recognizing that haze is a 3D phenomenon, the authors augment the standard horizontal and vertical position encodings with a third dimension: haze density.

The Transmission-Aware 3D Position Embedding incorporates horizontal, vertical, and haze density information to improve spatial awareness.

Haze density is estimated using the Dark Channel Prior (DCP), a well-established heuristic in dehazing. By embedding this density information directly into the Transformer's spatial awareness, the model learns to attend to regions with similar haze levels, regardless of their physical distance in the 2D image plane. This physically-grounded approach allows the Transformer to process the image in a way that respects the underlying geometry of the hazy scene.

Experimental Results

DeHamer was evaluated on several benchmark datasets, including RESIDE (SOTS-Outdoor) and real-world datasets like NH-HAZE and Dense-Haze.

Quantitative Performance

The model achieved state-of-the-art (SOTA) results across all major metrics:

  • SOTS-Outdoor: DeHamer reached a PSNR of 35.18 dB, surpassing previous leading methods like MSBDN.
  • NH-HAZE: On this challenging non-homogeneous haze dataset, DeHamer achieved 20.66 dB PSNR, representing a significant improvement of over 2.5 dB compared to GridDehazeNet.

Qualitative Analysis

Visual comparisons demonstrate that DeHamer is particularly effective at removing haze in extremely dense regions where previous models often leave "haze residues." The combination of global modulation and 3D awareness results in cleaner edges and more natural color reproduction, especially in complex outdoor scenes.

Conclusion

DeHamer represents a significant step forward in single-image dehazing by effectively bridging the gap between CNNs and Transformers. By treating haze density as a fundamental coordinate via the TA-3DPE and employing a sophisticated feature modulation scheme, the model achieves superior performance on both synthetic and real-world data. Future work may involve extending this 3D-aware modulation approach to other spatially-variant restoration tasks, such as low-light enhancement or rain removal.

Summary updated

Abstract

Despite single image dehazing has been made promising progress with Convolutional Neural Networks (CNNs), the inherent equivariance and locality of convolution still bottleneck deharing performance. Though Transformer has occupied various computer vision tasks, directly leveraging Transformer for image dehazing is challenging: 1) it tends to result in ambiguous and coarse details that are undesired for image reconstruction; 2) previous position embedding of Transformer is provided in logic or spatial position order that neglects the variational haze densities, which results in the sub-optimal dehazlng performance. The key insight of this study is to investigate how to combine CNN and Transformer for image dehazing. To solve the feature inconsistency issue between Transformer and CNN, we propose to modulate CNN features via learning modulation matrices (i.e., coefficient matrix and bias matrix) conditioned on Transformer features instead of simple feature addition or concatenation. The feature modulation naturally inherits the global context modeling capability of Transformer and the local representation capability of CNN. We bring a haze density-related prior into Trans-former via a novel transmission-aware 3D position embedding module, which not only provides the relative position but also suggests the haze density of different spatial regions. Extensive experiments demonstrate that our method, DeHamer, attains state-of-the-art performance on several image dehazing benchmarks.
BibTeX
@inproceedings{guo2022image,
  author = {Chunle Guo and Qixin Yan and Saeed Anwar and Runmin Cong and Wenqi Ren and Chongyi Li},
  title = {{Image Dehazing Transformer with Transmission-Aware 3D Position Embedding}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.00572},
}
Figure