ConfMix: Unsupervised Domain Adaptation for Object Detection via Confidence-based Mixing

Giulio Mattolin, Luca Zanella, Elisa Ricci, Yiming WangZenodo 2022 · DOI · OpenAlexGitHub resource

Abstract

Unsupervised Domain Adaptation (UDA) for object detection aims to adapt a model trained on a source domain to detect instances from a new target domain for which annotations are not available. Different from traditional approaches, we propose ConfMix, the first method that introduces a sample mixing strategy based on region-level detection confidence for adaptive object detector learning. We mix the local region of the target sample that corresponds to the most confident pseudo detections with a source image, and apply an additional consistency loss term to gradually adapt towards the target data distribution. In order to robustly define a confidence score for a region, we exploit the confidence score per pseudo detection that accounts for both the detector-dependent confidence and the bounding box uncertainty. Moreover, we propose a novel pseudo labelling scheme that progressively filters the pseudo target detections using the confidence metric that varies from a loose to strict manner along the training. We perform extensive experiments with three datasets, achieving state-of-the-art performance in two of them and approaching the supervised target model performance in the other.

Introduction

Deep learning-based object detectors are notoriously sensitive to domain shift, where a model trained on a labeled source domain (e.g., synthetic data or clear weather) performs poorly on an unlabeled target domain (e.g., real-world data or adverse weather). While manually annotating data for every new environment is prohibitively expensive, Unsupervised Domain Adaptation (UDA) offers a solution by adapting models using only unlabeled target data. Existing UDA methods often rely on adversarial training or simple pseudo-labeling, which can be unstable or prone to error reinforcement. ConfMix addresses these issues by intelligently selecting and mixing high-confidence target regions into source images during training.

The ConfMix Framework

The core of ConfMix is a training pipeline that leverages both labeled source data and pseudo-labeled target data through a novel mixing strategy and consistency losses.

The ConfMix framework overview, illustrating the flow from source ($x_S$) and target ($x_T$) images to mixed samples ($x_M$) and the application of consistency losses ($L_{cons}$).

Confidence-based Region Mixing

A key innovation of ConfMix is the region-level mixing strategy. Instead of mixing entire images, the target image is divided into regions (e.g., quadrants). The model calculates the average confidence of pseudo-detections within each region and selects the most reliable region to be pasted onto a source image. This creates a synthetic training sample $x_M$ that contains ground-truth labels from the source and high-quality "hallucinated" labels from the target.

Visual representation of the region mixing process: identifying high-confidence regions in the target image and blending them with the source image to create a mixed sample.

Uncertainty-Aware Confidence Metric

To ensure the reliability of pseudo-labels, ConfMix moves beyond standard classification scores ($C_{det}$). It incorporates localization uncertainty ($C_{bbx}$) by modeling bounding boxes as Gaussians. The final combined confidence, $C_{comb} = C_{det} \cdot C_{bbx}$, allows the model to ignore pseudo-detections that have high classification scores but poor localization accuracy.

Progressive Pseudo-Labeling

To handle the "cold start" problem where initial model predictions are unreliable, ConfMix employs a progressive pseudo-labeling scheme. Training begins with a "loose" threshold based on $C_{det}$ to provide sufficient initial signal. As training progresses, the threshold becomes "strict," shifting weight toward the uncertainty-aware $C_{comb}$ to refine the model's performance with only the most reliable data.

Experimental Results

ConfMix was evaluated on three standard UDA benchmarks for autonomous driving:

  • Sim10K → Cityscapes: Adapting from synthetic to real-world data.
  • KITTI → Cityscapes: Adapting between different real-world datasets.
  • Cityscapes → FoggyCityscapes: Adapting from clear to foggy weather.

The method achieved state-of-the-art (SOTA) results for one-stage detectors, reaching 56.3% mAP on Sim10K → Cityscapes (+1.7% over previous SOTA) and 52.2% mAP on KITTI → Cityscapes (+3.7% over previous SOTA). On FoggyCityscapes, it achieved 40.8% mAP, coming within 1.5% of the "Oracle" performance. Notably, ConfMix maintains high efficiency, running at 76 FPS on an NVIDIA V100, making it suitable for real-time applications.

Conclusion

ConfMix introduces a robust and efficient approach to UDA for object detection. By combining confidence-based region mixing with uncertainty-aware pseudo-labeling and a progressive training curriculum, it effectively bridges the domain gap. Future work will explore more sophisticated region partitioning strategies and the application of these principles to other architectures and domains.

Summary updated

Abstract

Unsupervised Domain Adaptation (UDA) for object detection aims to adapt a model trained on a source domain to detect instances from a new target domain for which annotations are not available. Different from traditional approaches, we propose ConfMix, the first method that introduces a sample mixing strategy based on region-level detec- tion confidence for adaptive object detector learning. We mix the local region of the target sample that corresponds to the most confident pseudo detections with a source image, and apply an additional consistency loss term to gradually adapt towards the target data distribution. In order to robustly define a confidence score for a region, we exploit the confidence score per pseudo detection that accounts for both the detector-dependent confidence and the bounding box uncertainty. Moreover, we propose a novel pseudo la- belling scheme that progressively filters the pseudo target detections using the confidence metric that varies from a loose to strict manner along the training. We perform ex- tensive experiments with three datasets, achieving state-of-the-art performance in two of them and approaching the supervised target model performance in the other.
BibTeX
@inproceedings{mattolin2022confmix,
  author = {Giulio Mattolin and Luca Zanella and Elisa Ricci and Yiming Wang},
  title = {{ConfMix: Unsupervised Domain Adaptation for Object Detection via Confidence-based Mixing}},
  booktitle = {Zenodo},
  year = {2022},
  doi = {10.5281/zenodo.7330908},
}
Figure