TL;DR
This paper introduces PromptHaze, a novel approach to real-world image dehazing that leverages depth information from the "Depth Anything" model. By iteratively refining depth prompts and employing a controllable dehazing strength, PromptHaze achieves superior recovery of background details and reduction of haze compared to existing methods.
What Problem Does This Paper Address?
Real-world image dehazing is a notoriously difficult problem. Hazy images are challenging to restore accurately because the haze itself obscures underlying details and can vary significantly in density and color. While many methods exist, they often struggle with the diverse nature of real-world haze, especially in dense fog scenarios. Existing deep learning approaches, while powerful, often fail to generalize well to real-world data due to limitations in synthetic training datasets. Furthermore, collecting large-scale, paired real-world hazy and clear image datasets is prohibitively expensive and time-consuming. This gap necessitates methods that can effectively handle real-world haze without relying on extensive paired data.
Methods
PromptHaze introduces a novel "prompt-by-prompt" dehazing strategy. The core idea is to leverage the "Depth Anything" model, a powerful foundational model for depth estimation, to provide a depth prompt. This depth prompt is not used directly but is iteratively refined.
Here's a breakdown of the key techniques:
- Depth Prompting: The "Depth Anything" model provides an initial depth map, which is then used as a "visual prompt" for the dehazing network. This is a novel application of depth estimation for this task.
- Controllable Dehazing Strength: The authors introduce a "dehazing strength factor" ($\alpha$) that allows the network to learn to control the degree of haze removal. This is achieved by synthesizing a new target image $I_\alpha = \alpha \cdot J_{gt} + (1 - \alpha) \cdot I_{syn}$, where $J_{gt}$ is the ground truth clear image and $I_{syn}$ is a synthetically generated hazy image.
- Iterative Refinement (Prompt-by-Prompt Strategy): Instead of a single dehazing pass, PromptHaze iteratively updates the depth prompt using the output of the previous dehazing step. This progressive refinement helps to mitigate errors in the initial depth estimation caused by severe haze and leads to more stable and accurate background recovery.
- Haze-Relevant Parameter Regression: The model also learns to predict haze-related parameters (like atmospheric light and haze density) to better understand and model the haze degradation. These predicted parameters are then projected into the decoder using Adaptive Layer Normalization (AdaLN) to further aid the dehazing process.
- Loss Functions: The training objective combines:
- Basic Reconstruction Loss: Charbonnier loss for pixel-wise reconstruction.
- Contrastive Regularization Loss: To improve generalization.
- Haze-Relevant Parameters Regression Loss: To encourage learning of haze characteristics.
Datasets
- Training: The authors utilize a dataset of 500 clean images, similar to what was used in the RIDCP method. Hazy images for training are generated online using a pipeline that incorporates estimated depth maps and controllable haze parameters.
- Evaluation:
- RTTS Dataset: A large-scale dataset of over 4,000 real-world hazy images with diverse scenes, times, resolutions, and degradations.
- Fattal's Dataset: A smaller dataset of 31 classic real-world hazy cases.
Key Insights and Ideas
- Depth as a Powerful Prior for Dehazing: The paper demonstrates that depth information, especially from robust foundational models like "Depth Anything," can be a highly effective prior for real-world image dehazing, a direction that was previously underexplored.
- Iterative Refinement is Crucial for Challenging Haze: Directly using a depth map from a hazy image can lead to errors. The "prompt-by-prompt" iterative strategy is key to overcoming these errors by progressively refining both the depth prompt and the dehazed output.
- Controllable Dehazing Strength is Beneficial: The ability to control the dehazing strength ($\alpha$) not only allows for flexible output but also aids in the training process by providing a more nuanced target for the network.
- Foundational Models are Game-Changers: The success of PromptHaze highlights the potential of leveraging large, pre-trained foundational models (like "Depth Anything") for downstream tasks, even in challenging domains like real-world image restoration.
Key Results
- Quantitative Performance: PromptHaze achieves state-of-the-art results on the RTTS dataset, outperforming methods like PSD and RIDCP in BRISQUE and NIMA metrics (higher NIMA is better, lower BRISQUE is better). For example, it achieves a BRISQUE score of 14.624 and a NIMA score of 5.1141.
- Qualitative Performance: Visual comparisons on RTTS and Fattal datasets show that PromptHaze significantly improves haze removal, recovers finer details, and produces more natural-looking backgrounds compared to other methods, including RIDCP.
- User Study: A user study confirmed PromptHaze's superior performance, with users selecting its results as the best significantly more often than other methods, particularly for dense fog removal.
- Ablation Studies: The paper shows that each component of PromptHaze (depth prompt, contrastive regularization, prompt-by-prompt strategy, haze-relevant parameter projection) contributes to the overall performance improvement.
Limitations and Future Work
- Artifacts and Color Distortions: The authors acknowledge that in some challenging scenes, removing dense haze can still lead to undesirable artifacts and color distortions in the foreground (as shown in Figure 7).
- Speed: Due to the need for multiple inference steps in the iterative process, the current speed of PromptHaze is not suitable for real-time applications.
- Future Work: The authors hope future research will address the artifact issue and improve the method's speed for real-time applications.
Summary updated