Abstract
Varicolored haze caused by chromatic casts poses haze removal and depth estimation challenges. Recent learning-based depth estimation methods are mainly targeted at dehazing first and estimating depth subsequently from haze-free scenes. This way, the inner connections between colored haze and scene depth are lost. In this paper, we propose a real-time transformer for simultaneous single image Depth Estimation and Haze Removal (DEHRFormer). DEHRFormer consists of a single encoder and two task-specific decoders. The transformer decoders with learnable queries are designed to decode coupling features from the task-agnostic encoder and project them into clean image and depth map, respectively. In addition, we introduce a novel learning paradigm that utilizes contrastive learning and domain consistency learning to tackle weak-generalization problem for real-world dehazing, while predicting the same depth map from the same scene with varicolored haze. Experiments demonstrate that DEHRFormer achieves significant performance improvement across diverse varicolored haze scenes over previous depth estimation networks and dehazing approaches.
Introduction
In computer vision, haze removal (dehazing) and depth estimation are traditionally treated as separate, sequential problems. Most pipelines dehaze an image first and then pass the result to a depth estimation model. However, this approach has two major flaws: the loss of intrinsic physical connections between haze density and scene depth, and the inability to handle "varicolored" haze—haze with distinct chromatic casts due to lighting and atmospheric particles. Standard models often fail to remove these color shifts or produce distorted depth maps.
DEHRFormer addresses these issues by treating haze as a depth cue rather than just noise, preserving the mathematical relationship defined by the atmospheric scattering model. By unifying these tasks into a single framework, the model achieves robust performance on varicolored scenes in real-time.
Methodology
The DEHRFormer architecture is a unified transformer-based framework consisting of a single encoder and two task-specific decoders.
Coupling Learning Encoder
To maintain real-time performance, DEHRFormer avoids computationally expensive self-attention in the encoder. Instead, it utilizes a CNN-based encoder with Multi-Scale Feature Modeling (MSFM) blocks, achieving $O(N)$ efficiency.
Task-Specific Decoders
The decoders use learnable queries to "probe" the encoder's features:
- Dehazing Decoder: Employs "haze type queries" to adaptively learn and remove diverse color degradations.
- Depth Decoder: Operates serially to the dehazing process, leveraging cleaned features to estimate depth from a haze-free perspective while maintaining structural context.
Semi-supervised Contrastive Learning
To bridge the domain gap between synthetic training data and real-world scenes, the authors employ a contrastive learning strategy. Unannotated real-world hazy images (URHI) are used as "negative samples," teaching the model to push its predictions away from hazy states toward clean states. Additionally, Domain Consistency Learning ensures the model predicts identical depth maps for a scene regardless of haze color or density.
Experimental Results
The model was evaluated on the VHRDE (Varicolored Haze Scene Depth Estimation) dataset, which contains 8,000 paired images with varicolored haze and corresponding depth maps.
Performance Metrics
DEHRFormer outperforms existing state-of-the-art (SOTA) methods like NAFNet and PMNet:
- Dehazing: Achieved a PSNR of 23.42 dB and SSIM of 0.876, surpassing NAFNet by 0.41 dB.
- Depth Estimation: Achieved an RMSE of 0.286, significantly outperforming sequential pipelines like SDDE (0.299).
- Efficiency: The model processes images in 0.034 seconds (~29.4 FPS) on an NVIDIA RTX 3090, enabling real-time video applications.
Visual results demonstrate that DEHRFormer effectively removes color casts (e.g., blue or yellow tints) that traditional methods leave as residual haze.
Conclusion
DEHRFormer provides a highly efficient, unified solution for simultaneous dehazing and depth estimation. By leveraging contrastive learning and a hybrid CNN-Transformer architecture, it overcomes the limitations of sequential processing and the domain gap in real-world varicolored haze scenes. Future work will focus on optimizing the number of learnable queries and extending the model to handle other atmospheric degradations like heavy rain or night-time glare.
Summary updated