TL;DR
This paper introduces a novel "Attentive Deformable Transformer" (ADT) network specifically designed for aerial image dehazing. By combining deformable convolutions with multi-head self-attention and a new "spatially attentive offset extractor," the ADT effectively preserves fine textures and global context, outperforming existing methods on both synthetic and real-world aerial imagery.
What Problem Does This Paper Address?
Aerial imagery is crucial for applications like surveillance and disaster assessment, but atmospheric haze significantly degrades image quality, reducing visibility and impacting the accuracy of downstream tasks. Existing dehazing methods often struggle with aerial images due to their unique characteristics, such as capturing scenes from a distance and the presence of non-uniform haze. Many methods either rely on external priors or fail to adequately capture both local texture details and global scene dependencies, leading to suboptimal dehazing results, especially in preserving fine structures.
Methods
The core of the proposed solution is the Attentive Deformable Transformer Block (ADT Block). This block integrates two key innovations:
- Deformable Multi-head Self-Attention (DMSA): Instead of standard self-attention, DMSA uses deformable convolutions to extract queries, keys, and values. This allows the attention mechanism to adapt to the geometric variations present in the input features, leading to more contextually relevant attention.
- Spatially Attentive Offset Extractor: Traditional deformable convolutions can sometimes extract offsets that extend beyond relevant image regions, leading to feature propagation issues. This new module extracts offsets and modulation values using spatially attentive features, ensuring that the deformable convolution focuses on truly relevant image areas and thus producing more accurate texture-relevant offsets.
Additionally, the network incorporates Edge Boosting Skip Connections (EBSC). Unlike standard skip connections that simply pass features, EBSC explicitly extracts and passes high-frequency edge information from shallow layers to deeper layers. This is vital for aerial images, which often contain important edge-sensitive structures like roads and buildings, ensuring these details are preserved during dehazing.
The overall architecture is a U-Net-like structure built upon these ADT Blocks, with input projection and output projection layers.
Datasets
The paper utilizes two primary datasets for training and evaluation:
- SatelK Dataset: This dataset contains pairs of clean and degraded aerial images with varying densities of haze (thin, moderate, thick). The authors used 640 pairs for training and 45 pairs for testing per haze density, with data augmentation. This dataset mimics non-uniform haze found in aerial imagery.
- RICE Dataset: This dataset features haze-degraded aerial images covering diverse earth surfaces (urban, ocean, desert, mountains). The authors used 800 training pairs and 100 testing pairs, also with data augmentation. This dataset primarily focuses on uniform haze removal.
Key Insights and Ideas
- Geometric Adaptability is Key for Aerial Dehazing: The paper strongly emphasizes that simply applying standard image processing techniques or even general vision transformers is insufficient for aerial images. The unique geometric variations and distant viewpoints necessitate methods that can adapt to these specific characteristics.
- Deformable Convolutions + Attention = Synergy: The combination of deformable convolutions (for geometric adaptation) and multi-head self-attention (for global context) is a powerful approach. However, the paper highlights the need for attentive deformable convolutions to ensure the offsets are contextually relevant.
- Offsets Matter (A Lot): The "spatially attentive offset extractor" is a critical component. It demonstrates that the quality and relevance of the offsets generated for deformable convolutions directly impact the final dehazing performance, preventing the propagation of irrelevant features.
- Don't Forget the Edges: Standard skip connections are insufficient for preserving fine details in aerial images. The EBSC module proves that explicitly handling edge information is crucial for high-quality restoration.
Key Results
- Quantitative Superiority: The proposed method consistently outperforms existing state-of-the-art methods on both the SatelK and RICE datasets across various haze densities. For instance, on the SatelK dataset (moderate haze), the proposed method achieves a PSNR of 27.03 and SSIM of 0.9472, surpassing methods like Huang et al. (25.31 PSNR, 0.9264 SSIM) and SkyGAN (25.58 PSNR, 0.9035 SSIM). On the RICE dataset, it achieves a PSNR of 33.79 and SSIM of 0.9703, outperforming UFormer (30.17 PSNR, 0.9531 SSIM).
- Qualitative Excellence: Visual comparisons (Figures 4, 5, and 6) show that the proposed method produces significantly clearer and more detailed dehazed images, effectively preserving textures, colors, and structures that are lost or distorted by other methods.
- Computational Efficiency: Table 3 indicates that the proposed method achieves competitive performance with a reasonable number of parameters and FLOPs, and a fast run-time per image.
Limitations and Future Work
- Noise Susceptibility: The paper acknowledges that aerial images can sometimes be noisy, and while the proposed method performs better than existing ones in such scenarios (Figure 10), it cannot completely remove noise.
- Future Work Suggestion: The authors suggest that training the network on images with combined hazy and noisy degradations could improve performance in handling such complex scenarios. They also mention that the proposed network's architecture could be adapted for other image restoration tasks beyond aerial image dehazing.
Summary updated