TL;DR
This paper introduces RPD-Diff, a novel diffusion model for image dehazing that tackles the challenging problem of dense and non-uniform haze. By incorporating physics-guided intermediate state targeting and a haze-aware timestep predictor, RPD-Diff achieves state-of-the-art performance in restoring clear images from heavily hazed inputs, outperforming existing methods in both quantitative metrics and visual quality.
What Problem Does This Paper Address?
Single-image dehazing aims to remove atmospheric haze from an image to reveal the underlying scene. While significant progress has been made, existing methods struggle with two key challenges: dense haze and non-uniform haze. Dense haze severely degrades image information, making it difficult for models to infer the original scene. Non-uniform haze, where haze density varies across the image, further complicates the task by requiring spatially adaptive processing. Traditional diffusion models, while powerful for image generation, often suffer from insufficient conditioning signals in dense haze scenarios and lack the ability to adapt to spatially varying haze, leading to suboptimal restoration, loss of details, and color distortions.
Methods
RPD-Diff builds upon the Denoising Diffusion Probabilistic Models (DDPMs) framework but introduces two core innovations:
- Physics-guided Intermediate State Targeting (PIST): Instead of directly generating a clear image from noise, PIST reformulates the diffusion process. It guides the model to first reconstruct intermediate "hazy-like" states based on physical haze properties (encoded in a transmission map). This strategy simplifies the initial denoising steps, especially in dense haze, by grounding the model in more reliable, albeit degraded, information. The intermediate target state is adaptively interpolated between the hazy and clear images, controlled by a weight function that considers the diffusion timestep and local haze density.
- Haze-Aware Denoising Timestep Predictor (HADTP): To address non-uniform haze, HADTP dynamically adjusts the denoising timestep for individual image patches. It uses a transmission map cross-attention mechanism to estimate a patch-specific timestep offset. This allows the model to allocate more denoising effort to heavily hazed regions and less to clearer ones, ensuring spatially adaptive restoration.
The framework also employs a patch-based diffusion approach to efficiently handle high-resolution images and facilitate localized processing. Noise estimates from overlapping patches are adaptively combined using learnable weights.
Datasets
The paper evaluates RPD-Diff on four widely-used real-world image dehazing benchmarks:
- Dense-Haze [2]
- NH-Haze [3]
- I-Haze [1]
- O-Haze [4]
These datasets collectively cover a range of challenging scenarios with dense and non-uniform haze in both indoor and outdoor environments. The authors designate the final five images from each dataset for testing and use the remaining for training.
Key Insights and Ideas
- Physics as a Guide for Diffusion: The paper demonstrates that incorporating physical priors (like haze density from transmission maps) into the diffusion process can significantly improve performance, especially in challenging conditions where traditional data-driven approaches falter.
- Intermediate Reconstruction is Easier: Reformulating the difficult task of direct clear image generation into a series of intermediate reconstruction steps (hazy-to-hazy-like) proves to be a more robust strategy for diffusion models when dealing with severe information loss.
- Spatially Adaptive Timesteps are Crucial: Non-uniform haze necessitates a dynamic approach. Dynamically adjusting denoising effort per patch, guided by local haze characteristics, is key to achieving coherent and artifact-free restoration.
- Patch-based Processing for Efficiency: Applying diffusion models in a patch-based manner offers a practical solution for handling high-resolution images, reducing computational complexity without sacrificing quality.
Key Results
- State-of-the-Art Performance: RPD-Diff consistently outperforms competing methods across all four benchmark datasets in both distortion-based (PSNR, SSIM) and perceptual (LPIPS, FID) metrics.
- Significant Gains in Dense/Non-Uniform Haze: The model shows a notable advantage in challenging dense and non-uniform haze scenarios. For instance, it surpasses the second-best method by up to 3.02 dB in PSNR on the Dense-HAZE dataset.
- Superior Visual Quality: Qualitative results show that RPD-Diff produces high-quality, haze-free images with enhanced detail clarity, improved color fidelity, and minimal artifacts, outperforming other methods that often leave residual haze or introduce distortions.
- Efficiency: RPD-Diff achieves the lowest inference time and FLOPs among compared diffusion-based methods, attributed to its patch-based approach.
Limitations and Future Work
- Transmission Map Estimation: The performance of PIST relies on an accurate estimation of the transmission map. While the paper proposes an improved estimation strategy with sky preservation, highly complex or unusual atmospheric conditions might still pose challenges.
- Computational Cost of Diffusion Models: Although RPD-Diff is more efficient than other diffusion models, diffusion models in general are still computationally intensive compared to some traditional methods.
The paper's success in addressing dense and non-uniform haze suggests that further exploration of physics-guided diffusion models for various image restoration tasks under adverse conditions could be a promising future direction. The adaptive nature of RPD-Diff also hints at potential applications in real-time dehazing systems.
Summary updated