HDCFN: Haze Distribution-aware Cross-modal Fusion Network for Infrared-guided Dense Haze Removal in UAVs

Junwei Zhao, Qianchun Luo, Shiliang Zhang, Shen Gao, Jie Wu

TL;DR

This paper introduces HDCFN, a novel network for removing dense haze from UAV imagery by fusing visible and infrared data. It leverages infrared's ability to penetrate haze for structural guidance and a haze-aware fusion mechanism to adaptively prioritize information, significantly improving the recovery of small, obscured objects.

What Problem Does This Paper Address?

Dense haze is a major challenge for Unmanned Aerial Vehicle (UAV) applications like surveillance and remote sensing. It severely degrades visible light imagery, obscuring small, distant ground targets and making them difficult to detect or analyze. While infrared imaging can penetrate haze better and preserve structural information, it lacks the color and fine texture details present in visible images. Existing single-image dehazing methods struggle with dense haze, and current visible-infrared fusion techniques are often designed for general purposes, not specifically for the demanding task of haze removal in dynamic UAV scenarios. This paper aims to bridge this gap by developing a method that effectively combines the strengths of both visible and infrared modalities for robust dense haze removal in UAV imagery.

Methods

The proposed Haze Distribution-aware Cross-modal Fusion Network (HDCFN) consists of two main components:

  1. Infrared-guided Multiscale Feature Enhancement (IMFE): This module extracts features from both visible and infrared images at multiple scales. Crucially, it uses the haze-resistant structural cues from infrared features to guide the refinement of visible features. This hierarchical approach helps in recovering fine details and small objects that are often lost in dense haze. The process involves downsampling features, using a channel attention mechanism guided by infrared features to enhance visible features, and then upsampling with skip connections.
  2. Haze Distribution-aware Cross-modal Adaptive Fusion (CMAF): This module adaptively fuses the enhanced visible and infrared features. It incorporates a Haze Distribution Estimator (HDE) that generates a haze density map. This map allows the CMAF module to dynamically adjust the weighting of visible and infrared features based on the local haze intensity. In regions with high haze, it prioritizes infrared features for structural integrity, while in clearer areas, it leverages visible features for color and texture. This adaptive fusion is key to handling the spatially varying nature of haze in UAV scenarios.

The model is trained using a composite loss function that includes a reconstruction loss (Mean Absolute Error) to ensure the output matches the ground truth and a cross-modal perception loss (Mean Squared Error) to promote alignment between visible and infrared features early in the network.

Datasets

The authors evaluated their method on three datasets:

  • VTUAV [42]: A publicly available dataset featuring synchronized visible and thermal imagery for UAVs, covering diverse environments and objects.
  • CART [13]: A dataset with high-resolution paired RGB-T images captured in natural environments.
  • CityUAV: A new dataset collected by the authors using their own UAV in foggy urban environments, featuring synchronized RGB and thermal infrared data.

Key Insights and Ideas

  • Infrared as a "Haze-Penetrating Guide": The core idea is to use infrared's inherent advantage in haze penetration not just for fusion, but as an active guide to enhance visible features. This is a departure from simply combining features and instead uses one modality to "correct" or "inform" the other.
  • Haze Density Dictates Fusion Strategy: The realization that haze isn't uniform and that a static fusion approach is suboptimal. Dynamically adapting the fusion based on local haze density is crucial for effective dehazing, especially in dynamic UAV environments.
  • Complementary Strengths for Small Objects: Dense haze is particularly problematic for small objects. By combining the structural robustness of infrared with the detail-rich (though degraded) visible information, the network can better reconstruct these challenging targets.

Key Results

  • State-of-the-Art Performance: HDCFN significantly outperforms existing state-of-the-art dehazing and visible-infrared fusion methods on the VTUAV and CART datasets.
  • On VTUAV, it achieves an average PSNR of 31.57 and SSIM of 0.918, an improvement of 8.1% in PSNR and 4.2% in SSIM over the second-best method (VIFNet).
  • On CART, it shows an average PSNR gain of 7.9% and SSIM gain of 5.7% in dense haze conditions compared to VIFNet.
  • Real-World Validation: Experiments on the real-world CityUAV dataset demonstrate HDCFN's effectiveness, achieving superior perceptual quality (lower NIQE and PIQE scores) and visually more appealing results than competing methods.
  • Improved Downstream Task Performance: When applied to object detection on dehazed images, HDCFN-processed images lead to significantly improved detection accuracy (mAP) and recall, demonstrating the practical benefit of enhanced image clarity.
  • Ablation Studies: The IMFE and CMAF modules both contribute substantially to the overall performance, with the full HDCFN model achieving the best results. The perception loss term also plays a vital role in refining visual quality.

Limitations and Future Work

  • Limitations: The paper primarily focuses on dense haze. While it shows improvements in lighter haze conditions, its core strength lies in tackling severe degradation. The reliance on synchronized RGB-T data is also a prerequisite.
  • Future Work: The authors suggest exploring more advanced fusion strategies and investigating the method's applicability to other challenging atmospheric conditions beyond haze. Further research into unsupervised or weakly supervised dehazing using multimodal data could also be a promising direction.

Summary updated

Abstract

In UAV applications, dense haze severely obscures small ground-level objects, hindering the recovery of fine details. Existing visible-only dehazing methods struggle with such dense occlusions, while infrared imaging lacks color and fine texture information. To address these limitations, we propose the Haze Distribution-aware Cross-modal Fusion Network (HDCFN). HDCFN features two key components: (i) an infrared-guided multiscale feature enhancement framework that integrates haze-resistant structural cues from infrared modality with visible features across coarse to fine, improving the recovery of small objects, and (ii) a haze distribution-aware cross-modal fusion module that adaptively prioritizes relevant information from each modality according to haze density. This framework effectively combines the complementary strengths of visible and infrared imaging for dense haze removal. Extensive experiments on multiple public datasets show that HDCFN outperforms state-of-the-art dehazing and fusion methods, yielding higher-quality and more detailed images.
BibTeX
@inproceedings{zhao2025hdcfn,
  author = {Junwei Zhao and Qianchun Luo and Shiliang Zhang and Shen Gao and Jie Wu},
  title = {{HDCFN: Haze Distribution-aware Cross-modal Fusion Network for Infrared-guided Dense Haze Removal in UAVs}},
  booktitle = {ACM Multimedia},
  year = {2025},
  doi = {10.1145/3746027.3754994},
}
Figure