C2AIR: Consolidated Compact Aerial Image Haze Removal

Ashutosh Kulkarni, Shruti S. Phutke, Santosh Kumar Vipparthi, Subrahmanyam MuralaWACV 2024

TL;DR

This paper introduces C2AIR, a novel consolidated and compact deep learning model for aerial image dehazing. It addresses the limitations of existing methods that require separate models for different haze densities and are computationally expensive, by proposing a single, efficient model that generalizes across various haze levels.

What Problem Does This Paper Address?

Aerial images, crucial for applications like surveillance, urban planning, and disaster management, are often degraded by atmospheric haze. This haze reduces image contrast, color saturation, and sharpness, making it difficult to extract meaningful information. Existing dehazing methods for aerial imagery often fall into two categories: computationally heavy approaches that require separate models for light, moderate, and dense haze, or methods that are computationally expensive and don't generalize well across different haze conditions. This necessitates a more efficient and unified approach that can handle varying haze densities with a single model.

Methods

The core of C2AIR lies in three novel blocks designed to address the limitations of prior work:

  • Density Aware Query Modulated (DAQM) Block: This block is inspired by transformer architectures in natural language processing. It extracts density-specific information from the input image and uses it to modulate queries, allowing the network to adapt its restoration process to the specific haze density present. This is a key innovation for achieving a consolidated model.
  • Cross Collaborative Feed-Forward (CCF) Block: To handle the diverse structures and varying sizes of objects in aerial images, this block employs multi-kernel feature extractors. This "cross-collaborative" approach allows the network to effectively capture features at different receptive fields, leading to better perceptual quality.
  • Adaptive Gated Feature Fusion (AGFF) Block: This block addresses the challenge of fusing features from different streams (original scale and multi-scale) within the network. It employs a two-tiered attention mechanism for scale-level and intra-feature attention, ensuring that only relevant and non-degraded features are propagated, thus improving restoration quality.

The overall architecture is designed to be lightweight, with a focus on reducing computational complexity and the number of trainable parameters.

Datasets

The paper utilizes the following datasets for training and evaluation:

  • Sate-1K Dataset: This dataset contains pairs of clean and degraded aerial images with varying haze densities (thin, moderate, thick).
  • RICE Dataset: This dataset includes aerial images degraded by haze, covering diverse earth surfaces.

For consolidated training, the authors merged these two datasets, creating a combined training set of 2720 image pairs. Testing was performed on the respective test sets of each dataset.

Key Insights and Ideas

  • Consolidation is Key: The most significant insight is that a single, consolidated model can effectively handle diverse haze densities in aerial images, eliminating the need for multiple specialized models. This simplifies deployment and improves efficiency.
  • Density-Awareness Matters: Explicitly learning and utilizing haze density information (via DAQM) is crucial for adapting the restoration process and achieving robust performance across different haze levels.
  • Multi-Receptive Field Learning: Capturing features at various scales and receptive fields (via CCF) is essential for accurately restoring intricate structures in aerial imagery.
  • Intelligent Feature Fusion: Adaptive fusion of features from different processing streams (via AGFF) is critical to avoid propagating noise and degraded information, leading to cleaner and more accurate restorations.
  • Efficiency Gains: The proposed architecture achieves a significant reduction in parameters and computational complexity (GFLOPS) compared to existing methods, making it more practical for real-world applications.

Key Results

  • Quantitative Performance: C2AIR achieves state-of-the-art performance on both the Sate-1K and RICE datasets, outperforming existing methods in terms of PSNR and SSIM across all haze densities. For instance, on the Sate-1K dataset, the proposed consolidated method achieved a PSNR of 30.32 for thin haze, 29.65 for moderate haze, and 27.97 for thick haze.
  • Computational Efficiency: The proposed method is significantly more efficient, boasting approximately half the number of trainable parameters and about one-seventh of the GFLOPS compared to other methods. It also achieves a lower run-time per image.
  • Qualitative Results: Visual comparisons demonstrate that C2AIR effectively restores details and color balance in hazy aerial images, producing clearer and more visually appealing results than competing methods.
  • Generalizability: The paper demonstrates the applicability of C2AIR to other image restoration tasks, specifically underwater image enhancement, where it also shows superior performance, highlighting the robustness of its architectural design.

Limitations and Future Work

  • Limitations: While the paper focuses on aerial image dehazing, the authors acknowledge that further exploration into other image restoration tasks is warranted. The current study demonstrates applicability to underwater image enhancement, suggesting potential for broader use.
  • Future Work: The authors suggest exploring the applicability of their proposed architecture to other image restoration tasks. The success in underwater image enhancement indicates a promising direction for future research.

Summary updated

BibTeX
@inproceedings{kulkarni2024c,
  author = {Ashutosh Kulkarni and Shruti S. Phutke and Santosh Kumar Vipparthi and Subrahmanyam Murala},
  title = {{C2AIR: Consolidated Compact Aerial Image Haze Removal}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure