Abstract
Most object detection methods for autonomous driving usually assume a consistent feature distribution between training and testing data, which is not always the case when weathers differ significantly. The object detection model trained under clear weather might not be effective enough in foggy weather because of the domain gap. This paper proposes a novel domain adaptive object detection framework for autonomous driving under foggy weather. Our method leverages both image-level and object-level adaptation to diminish the domain discrepancy in image style and object appearance. To further enhance the model's capabilities under challenging samples, we also come up with a new adversarial gradient reversal layer to perform adversarial mining for the hard examples together with domain adaptation. Moreover, we propose to generate an auxiliary domain by data augmentation to enforce a new domain-level metric regularization. Experimental results on public benchmarks show the effectiveness and accuracy of the proposed method.
Introduction
Standard object detection models, such as Faster R-CNN, perform exceptionally well under clear skies. However, their performance plummets in foggy or hazy conditions. This performance drop is primarily due to a domain gap: the visual features learned during training in clear weather do not match the features encountered in foggy environments.
Manually labeling large datasets of foggy images is prohibitively expensive and time-consuming. Consequently, research has shifted toward Unsupervised Domain Adaptation (UDA)—transferring knowledge from a labeled "source" domain (clear weather) to an unlabeled "target" domain (foggy weather) to learn weather-invariant features. This paper introduces a framework that combines dynamic adversarial mining with a third "auxiliary" domain for metric regularization, significantly improving detection accuracy in low-visibility conditions without requiring any manually labeled foggy images.
Proposed Framework
The proposed architecture builds upon the Faster R-CNN framework, introducing three key innovations to bridge the domain gap.
1. Multi-level Adaptation
The framework employs domain classifiers at two distinct scales:
- Image-level Adaptation: Focuses on global image styles, such as illumination and scale.
- Object-level Adaptation: Focuses on local object appearance. These classifiers attempt to identify the domain of an image or object, while the detection backbone is trained to "confuse" them, thereby learning features that are invariant across different weather conditions.
2. Adversarial Gradient Reversal Layer (AdvGRL)
Traditional domain adaptation treats all training samples equally. However, some samples are "harder" to adapt than others. The authors proposed AdvGRL, which dynamically adjusts back-propagated gradients. When the domain classifier easily identifies a sample's domain (indicating a large gap), AdvGRL increases the adversarial response, forcing the model to focus more on these difficult edge cases.
3. Auxiliary Domain & Metric Regularization
A unique aspect of this research is the introduction of an auxiliary domain. By using data augmentation (specifically RainMix), the authors create a third domain (e.g., rainy weather). They then apply a triplet loss to ensure that features from the source (clear) and target (fog) domains are pulled closer together than they are to this third auxiliary domain. This provides a stronger geometric constraint on the feature space, improving the robustness of the learned representations.
Experimental Evaluation
The researchers evaluated their method using standard benchmarks including Cityscapes (source), Foggy Cityscapes (target), and KITTI (cross-camera adaptation).
The proposed method achieved a mean Average Precision (mAP) of 42.3% on the Foggy Cityscapes benchmark, a 1.2% improvement over previous state-of-the-art methods. Significant gains were observed in specific categories like "train" (48.7% AP) and "bus." In cross-camera adaptation tests (Cityscapes to KITTI), the model achieved a 74.38 AP for cars, demonstrating strong generalization capabilities across different hardware setups.
Conclusion
This work demonstrates that focusing on hard example mining and utilizing auxiliary domains can significantly enhance unsupervised domain adaptation for object detection. By addressing the domain gap at both image and object levels, the framework allows autonomous vehicles to maintain high detection accuracy in adverse weather. Future work may involve extending this framework to other challenging conditions like nighttime or heavy snow and refining the metric regularization for real-world datasets where pixel-to-pixel correspondence is not available.
Summary updated