Abstract
Adverse weather conditions such as haze and rain corrupt the quality of captured images, which cause detection networks trained on clean images to perform poorly on these corrupted images. To address this issue, we propose an unsupervised prior-based domain adversarial object detection framework for adapting the detectors to hazy and rainy conditions. In particular, we use weather-specific prior knowledge obtained using the principles of image formation to define a novel prior-adversarial loss. The prior-adversarial loss, which we use to supervise the adaptation process, aims to reduce the weather-specific information in the features, thereby mitigating the effects of weather on the detection performance. Additionally, we introduce a set of residual feature recovery blocks in the object detection pipeline to de-distort the feature space, resulting in further improvements. Evaluations performed on various datasets (Foggy-Cityscapes, Rainy-Cityscapes, RTTS and UFDD) for rainy and hazy conditions demonstrates the effectiveness of the proposed approach.
Introduction
Standard object detection models, such as those used in autonomous driving, are typically trained on "clean" images. When these models encounter haze or rain, their performance often plummets because weather introduces a "domain shift"—the pixels look fundamentally different than what the model expects.
Common solutions usually fall into two categories: pre-processing (using a separate dehazing or deraining network) or generic Domain Adaptation (DA). Pre-processing is often computationally expensive and can introduce artifacts, while generic DA methods often assume degradation is uniform across the image. This paper proposes a physics-informed approach that uses weather-specific priors to guide the adaptation process, ensuring the detector remains robust in non-uniform adverse conditions.
Methodology
The proposed framework is built upon the Faster R-CNN architecture and introduces three novel components to handle weather-induced distortions.
As illustrated in the architecture above, the system includes:
- Prior Estimation Network (PEN): Instead of a simple binary classifier, the PEN predicts weather-specific information. For haze, it predicts a transmission map (how much light gets through); for rain, it predicts rain residue (the streaks themselves).
- Prior-Adversarial Loss (PAL): This is the core of the adversarial setup. The PEN tries to predict the weather prior from image features, while the feature extractor tries to "fool" the PEN. This forces the network to discard weather-specific information, creating weather-invariant features.
- Residual Feature Recovery Blocks (RFRB): These specialized blocks act as a "repair kit" in the target pipeline, de-distorting features before they reach the final detection stage.
Physics-Informed Prior Adaptation
The most significant insight of this work is that weather follows mathematical models (like the Atmospheric Scattering Model) rather than being random noise. Using these models as "priors" provides a much stronger training signal than generic domain adaptation.
Unlike standard adversarial methods that apply a single label to an entire image, weather priors are spatially varying. As shown in Figure 1, this allows the model to learn that different parts of an image—such as the foreground versus the background—are affected by weather differently. Crucially, because these priors can be calculated mathematically, the entire process is unsupervised, requiring no human-labeled weather data.
Experimental Results
The researchers evaluated their model on several challenging benchmarks, consistently outperforming both "Source Only" baselines and state-of-the-art domain adaptation methods:
- Foggy-Cityscapes: Achieved 39.3 mAP (mean Average Precision), a significant jump over the 24.4 mAP baseline.
- Rainy-Cityscapes: Reached 36.1 mAP on a newly introduced dataset featuring artificial rain streaks with varying noise levels.
- Real-World Performance: On the RTTS dataset, the model achieved 34.1 mAP, proving that the priors learned from synthetic data translate effectively to the messy reality of actual fog and rain.
Conclusion
By integrating the physics of haze and rain directly into the training process, this framework allows object detectors to "see" through adverse weather without additional inference overhead. While the current work treats haze and rain as separate tasks, future research could focus on a universal adapter for multiple simultaneous conditions and deployment on more modern, lightweight architectures.
Summary updated