Domain Adaptive Object Detection for Autonomous Driving under Foggy Weather

Jinlong Li, Runsheng Xu, Jin Ma, Qin Zou, Jiaqi Ma, Hongkai Yu

Introduction

Autonomous driving systems rely heavily on robust object detection to ensure safety. While modern detectors like Faster R-CNN perform exceptionally well in clear weather, their performance degrades significantly in adverse conditions such as fog. This degradation is caused by a "domain gap"—the discrepancy between the visual features of the training data (source domain) and the real-world deployment environment (target domain). Since manual annotation of foggy images is expensive, Unsupervised Domain Adaptation (UDA) has emerged as a vital research area. This paper presents a framework that bridges this gap by learning domain-invariant features without requiring target-domain labels.

Methodology

The proposed framework extends the Faster R-CNN architecture with a multi-level adaptation strategy and a novel regularization technique.

Multi-Level Adaptation and AdvGRL

The model performs adaptation at two scales: the image level, which captures global context like illumination and style, and the object level, which focuses on the local appearance of specific instances. To enhance this process, the authors introduce the Adversarial Gradient Reversal Layer (AdvGRL). Unlike standard GRLs that treat all samples equally, AdvGRL implements adversarial mining. It identifies "hard examples"—samples that the domain classifier can easily distinguish—and assigns them higher weights. This forces the backbone network to prioritize making these challenging features domain-invariant.

Auxiliary Domain and Metric Regularization

A key innovation is the use of an auxiliary domain created via data augmentation (e.g., RainMix). By introducing a third "rainy" domain, the authors can apply domain-level metric regularization. Using a triplet loss constraint, the model ensures that the feature distance between the clear (source) and foggy (target) domains is minimized relative to the distance to the rainy (auxiliary) domain. This structured feature space prevents overfitting to specific target-domain artifacts and promotes better generalization.

The proposed domain adaptation framework featuring image-level and object-level classifiers, AdvGRL, and an auxiliary domain for metric regularization.

Experimental Results

The framework was evaluated on the Foggy Cityscapes and KITTI datasets, using standard Cityscapes as the source domain.

Quantitative Performance

The method achieved a state-of-the-art 42.3% mean Average Precision (mAP) on Foggy Cityscapes, a significant improvement over the 23.41% mAP of a non-adapted baseline. Notable gains were observed in difficult categories such as buses (51.2% AP) and trains (48.7% AP). In cross-camera adaptation tests (Cityscapes to KITTI), the model reached 74.38% AP for cars, demonstrating its robustness to simultaneous changes in camera hardware and environmental conditions.

Qualitative Analysis

Visual comparisons highlight the effectiveness of the adaptation. While baseline models often miss objects obscured by haze or produce false positives, the proposed method successfully detects distant vehicles and pedestrians in thick fog.

Qualitative comparison of object detection: (a) source domain (clear), (b) baseline on target domain (foggy), and (c) proposed method on target domain (foggy).

Conclusion

By combining adversarial hard-example mining with multi-domain metric regularization, this work significantly advances the state-of-the-art in foggy weather object detection. The framework's ability to learn robust features without target labels makes it highly practical for real-world autonomous driving. Future research will explore adaptation to even more diverse conditions, such as nighttime fog, and the integration of multi-modal sensor data like LiDAR.

Summary updated

Abstract

Most object detection methods for autonomous driving usually assume a consistent feature distribution between training and testing data, which is not always the case when weathers differ significantly. The object detection model trained under clear weather might be not effective enough on the foggy weather because of the domain gap. This paper proposes a novel domain adaptive object detection framework for autonomous driving under foggy weather. Our method leverages both image-level and object-level adaptation to diminish the domain discrepancy in image style and object appearance. To further enhance the model’s capabilities under challenging samples, we also come up with a new adversarial gradient reversal layer to perform adversarial mining for the hard examples together with domain adaptation. Moreover, we propose to generate an auxiliary domain by data augmentation to enforce a new domain-level metric regularization. Experimental results on public benchmarks show the effectiveness and accuracy of the proposed method. The code is available at https://github.com/jinlong17/DA-Detect.
BibTeX
@inproceedings{li2023domain,
  author = {Jinlong Li and Runsheng Xu and Jin Ma and Qin Zou and Jiaqi Ma and Hongkai Yu},
  title = {{Domain Adaptive Object Detection for Autonomous Driving under Foggy Weather}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  doi = {10.1109/wacv56688.2023.00068},
}
Figure