Abstract
Unsupervised Domain Adaptation (UDA) for object detection aims to adapt a model trained on a source domain to detect instances from a new target domain for which annotations are not available. Different from traditional approaches, we propose ConfMix, the first method that introduces a sample mixing strategy based on region-level detection confidence for adaptive object detector learning. We mix the local region of the target sample that corresponds to the most confident pseudo detections with a source image, and apply an additional consistency loss term to gradually adapt towards the target data distribution. In order to robustly define a confidence score for a region, we exploit the confidence score per pseudo detection that accounts for both the detector-dependent confidence and the bounding box uncertainty. Moreover, we propose a novel pseudo labelling scheme that progressively filters the pseudo target detections using the confidence metric that varies from a loose to strict manner along the training. We perform extensive experiments with three datasets, achieving state-of-the-art performance in two of them and approaching the supervised target model performance in the other.
Introduction
Deep learning-based object detectors are notoriously sensitive to domain shift, where a model trained on a labeled source domain (e.g., synthetic data or clear weather) performs poorly on an unlabeled target domain (e.g., real-world data or adverse weather). While manually annotating data for every new environment is prohibitively expensive, Unsupervised Domain Adaptation (UDA) offers a solution by adapting models using only unlabeled target data. Existing UDA methods often rely on adversarial training or simple pseudo-labeling, which can be unstable or prone to error reinforcement. ConfMix addresses these issues by intelligently selecting and mixing high-confidence target regions into source images during training.
The ConfMix Framework
The core of ConfMix is a training pipeline that leverages both labeled source data and pseudo-labeled target data through a novel mixing strategy and consistency losses.
Confidence-based Region Mixing
A key innovation of ConfMix is the region-level mixing strategy. Instead of mixing entire images, the target image is divided into regions (e.g., quadrants). The model calculates the average confidence of pseudo-detections within each region and selects the most reliable region to be pasted onto a source image. This creates a synthetic training sample $x_M$ that contains ground-truth labels from the source and high-quality "hallucinated" labels from the target.
Uncertainty-Aware Confidence Metric
To ensure the reliability of pseudo-labels, ConfMix moves beyond standard classification scores ($C_{det}$). It incorporates localization uncertainty ($C_{bbx}$) by modeling bounding boxes as Gaussians. The final combined confidence, $C_{comb} = C_{det} \cdot C_{bbx}$, allows the model to ignore pseudo-detections that have high classification scores but poor localization accuracy.
Progressive Pseudo-Labeling
To handle the "cold start" problem where initial model predictions are unreliable, ConfMix employs a progressive pseudo-labeling scheme. Training begins with a "loose" threshold based on $C_{det}$ to provide sufficient initial signal. As training progresses, the threshold becomes "strict," shifting weight toward the uncertainty-aware $C_{comb}$ to refine the model's performance with only the most reliable data.
Experimental Results
ConfMix was evaluated on three standard UDA benchmarks for autonomous driving:
- Sim10K → Cityscapes: Adapting from synthetic to real-world data.
- KITTI → Cityscapes: Adapting between different real-world datasets.
- Cityscapes → FoggyCityscapes: Adapting from clear to foggy weather.
The method achieved state-of-the-art (SOTA) results for one-stage detectors, reaching 56.3% mAP on Sim10K → Cityscapes (+1.7% over previous SOTA) and 52.2% mAP on KITTI → Cityscapes (+3.7% over previous SOTA). On FoggyCityscapes, it achieved 40.8% mAP, coming within 1.5% of the "Oracle" performance. Notably, ConfMix maintains high efficiency, running at 76 FPS on an NVIDIA V100, making it suitable for real-time applications.
Conclusion
ConfMix introduces a robust and efficient approach to UDA for object detection. By combining confidence-based region mixing with uncertainty-aware pseudo-labeling and a progressive training curriculum, it effectively bridges the domain gap. Future work will explore more sophisticated region partitioning strategies and the application of these principles to other architectures and domains.
Summary updated