Cross-domain object detection based foggy image enhancement

Qiang Guo, Shiliang Pu, Shifeng Zhang, Bo LiJournal of Image and Graphics 2022 · DOI

Abstract

Outdoor surveillance systems often suffer from reduced image clarity and object saliency due to fog and haze. Features extracted from foggy images—whether for human visual quality or machine-based object detection—differ significantly from those in clear images. This paper proposes an unsupervised prior-hybrid image feature-level enhancement network. By combining traditional atmospheric scattering models with a deep feature-level enhancement module, the system bridges the domain gap without requiring labeled foggy data. Experimental results on real-world datasets like RTTS and Foggy Driving Dense show significant improvements in both visual quality (50.83% increase in gradient ratio) and detection accuracy (6.40% increase in mAP), while maintaining real-time performance at 50 frames per second.

The Challenge of Foggy Surveillance

Outdoor surveillance systems frequently struggle with fog, mist, and haze. These conditions do not just make images look "blurry" to humans; they fundamentally alter the statistical distribution of pixels, affecting saturation, contrast, and contours. This creates a significant "domain gap": an object detector trained on clear images often fails when deployed in foggy environments because the features it relies on have been distorted.

The core challenge in addressing this is two-fold. First, there is a severe scarcity of paired data; it is nearly impossible to capture perfectly synchronized "foggy" and "clear" images of the same real-world scene for supervised training. Second, there is a disconnect between human and machine perception. Traditional dehazing methods focus on making images look good to the human eye, which does not always translate to better performance for automated detection algorithms like Faster R-CNN.

A Hybrid Unsupervised Methodology

To overcome these hurdles, the proposed method utilizes an unsupervised prior-hybrid network consisting of two main stages:

  1. Traditional Prior-Based Fog Inversion: Instead of relying solely on deep learning, the network uses an extended atmospheric scattering model. This module estimates transmission and illumination to perform an initial "coarse" dehazing. This provides a physically grounded starting point that stabilizes the subsequent learning process.
  2. Feature-Level Enhancement Module: This deep learning component, based on a Conditional Generative Adversarial Network (CGAN), treats the coarsely dehazed image as input and refines it. Unlike standard pixel-level enhancers, this module uses a target semantic discriminator to analyze domain differences at the feature level. This ensures that the enhanced image retains the specific textures and outlines that object detectors rely on.
The proposed method aims to enhance low-quality images to improve both visual clarity and machine-readability.

Bridging the Feature Gap

A key insight of this research is that pixel-level improvement does not equal feature-level improvement. Improving the global contrast of an image can sometimes introduce artifacts that confuse a detector. By focusing on feature-level enhancement, the network prioritizes "machine-friendly" details like sharp edges and semantic consistency. The inclusion of a semantic discriminator essentially instructs the generator to not just make the image look clear, but to make it look like something a detector can recognize.

Experimental Results and Real-World Impact

The proposed method was evaluated against five state-of-the-art methods using benchmarks such as the RTTS (Real-world Task-driven Testing Set) and Foggy Driving Dense datasets. The results demonstrate a clear advantage:

  • Detection Accuracy: On the RTTS dataset, the Mean Average Precision (mAP) improved by 6.40% and Recall by 7.79% compared to the next best algorithm.
  • Visual Quality: The gradient ratio (R-value), measuring edge restoration, saw a massive average increase of 50.83%. The perceptual quality metric (IL-NIQE) improved by 6.33%.
  • Efficiency: The system processes high-definition video at 50 frames per second (fps), making it highly viable for real-time surveillance and autonomous vehicle applications.

Conclusion and Future Directions

This hybrid approach successfully satisfies the dual requirements of human visual monitoring and automated cross-domain object detection in foggy weather. While the results are impressive, future work will focus on improving texture recovery in extremely dense fog and ensuring temporal consistency for smoother video processing. Extending this framework to handle other complex weather conditions, such as heavy rain or snow, remains a primary goal for future iterations.

Summary updated

Abstract

目的室外监控在雾霾天气所采集图像的成像清晰度和目标显著程度均会降低,当在雾霾图像提取与人眼视觉质量相关的自然场景统计特征和与目标检测精度相关的目标类别语义特征时,这些特征与从清晰图像提取的特征存在明显差别。为了提升图像质量并且在缺乏雾霾天气目标检测标注数据的情况下提升跨域目标检测效果,本文综合利用传统方法和深度学习方法,提出了一种无监督先验混合图像特征级增强网络。方法利用本文提出的传统先验构成雾气先验模块;其后连接一个特征级增强网络模块,将去散射图像视为输入图像,利用像素域和特征域的损失实现场景统计特征和目标类别语义相关表观特征的增强。该混合网络突破了传统像素级增强方法难以表征抽象特征的制约,同时克服了对抗迁移网络难以准确衡量无重合图像域在特征空间分布差异的弱点,也减弱了识别算法对于低能见度天候采集图像标注数据的依赖,可以同时提高雾霾图像整体视觉感知质量以及局部目标可识别表现。结果实验在两个真实雾霾图像数据集、真实图像任务驱动的测试数据集(real-world task-driven testing set, RTTS)和自动驾驶雾天数据集(foggy driving dense)上与最新的5种散射去除方法进行了比较,相比于各指标中性能第2的算法,本文方法结果中梯度比指标R值平均提高了50.83%,属于感知质量指标的集成自然图像质量评价指标(integrated local natural image quality evaluator, IL-NIQE)值平均提高了6.33%,属于跨域目标检测指标的平均精准率(mean average precision, MAP)值平均提高了6.40%,平均查全率Recall值平均提高了7.79%。实验结果表明,本文方法结果在视觉质量和目标可识别层面都优于对比方法,并且本文方法对于高清视频的处理速度达50帧/s,且无需标注数据,因而在监控系统具有更高的实用价值。结论本文方法可以同时满足雾霾天候下对采集视频进行人眼观看和使用识别算法进行跨域目标检测的需求,具有较强的应用意义。
BibTeX
@article{guo2022cross,
  author = {Qiang Guo and Shiliang Pu and Shifeng Zhang and Bo Li},
  title = {{Cross-domain object detection based foggy image enhancement}},
  journal = {Journal of Image and Graphics},
  year = {2022},
  doi = {10.11834/jig.210994},
}
Figure