TL;DR
This paper tackles the challenge of adapting object detection models to new domains with limited or no labeled data. They propose a novel approach combining Feature Distribution Alignment (FDA) with a robust Self-Training (ST) method that incorporates Mixup regularization to improve accuracy and stability, outperforming existing methods on several benchmarks.
What Problem Does This Paper Address?
Training deep learning models for object detection typically requires large, meticulously annotated datasets. However, when a model trained on one dataset (source domain) is applied to a different dataset with a different visual style or characteristics (target domain), its performance often degrades significantly due to a "domain gap." Collecting and annotating new data for every target domain is prohibitively expensive and time-consuming. Unsupervised Domain Adaptation (UDA) aims to bridge this gap by leveraging unlabeled target domain data. Existing UDA methods for object detection often struggle with large domain shifts or can be destabilized by noisy pseudo-labels generated during self-training. This paper addresses these limitations by proposing a more robust and effective UDA strategy for one-stage object detectors.
Methods
The proposed method combines two main components:
- Feature Distribution Alignment (FDA): This step aims to reduce the domain gap by aligning the feature distributions of the source and target domains. It uses an adversarial training approach with domain classifiers attached to different layers of the feature extractor. This helps the model learn domain-invariant representations, which in turn leads to more accurate pseudo-labels for the target domain. The paper adapts existing FDA techniques for two-stage detectors to the one-stage SSD architecture.
- Self-Training with Mixup (STM): This is the core of their novel approach.
- Pseudo-Label Generation with Periodic Data Distillation (PDD): To further improve the quality of pseudo-labels generated from the FDA-aligned model, they employ PDD. This involves applying the model to multiple transformed versions of a target image and then ensembling the predictions to generate a more reliable pseudo-label. This process is periodically updated as the model improves.
- Mixup Regularization: To combat the instability and negative impact of potentially incorrect pseudo-labels during self-training, they introduce Mixup regularization. Unlike standard Mixup in classification, they apply it to object class labels. By increasing the interpolation strength (controlled by the hyperparameter 'a'), they make it harder for the model to memorize incorrect labels and encourage more consistent predictions, especially for samples near decision boundaries.
The overall training process involves an initial phase of FDA followed by the STM phase.
Datasets
The paper uses the following datasets:
- Source Domain: PASCAL VOC (2007 and 2012 training and validation splits)
- Target Domains:
- Clipart1k
- Comic2k
- Watercolor2k
- Real-world Scenario: COCO (source) to Caltech (target) for pedestrian detection.
Key Insights and Ideas
- FDA as a Precursor to Reliable Pseudo-Labels: The paper highlights that while FDA alone can reduce domain shifts, its primary benefit in this context is enabling the generation of more accurate pseudo-labels for subsequent self-training.
- The Double-Edged Sword of Self-Training: They acknowledge that self-training is powerful but highly susceptible to errors in pseudo-labels, which can cascade and degrade performance.
- Mixup's Crucial Role in Stabilizing Self-Training: The introduction of Mixup regularization is presented as a key innovation that significantly stabilizes the self-training process, preventing performance degradation even with prolonged training.
- Adapting Mixup for Object Detection: A notable detail is their adaptation of Mixup from image classification to object detection by applying it to class labels rather than entire images or bounding box coordinates.
- PDD for Enhanced Pseudo-Label Accuracy: Periodic Data Distillation is shown to be beneficial in generating more robust pseudo-labels, especially when domain shifts are significant.
Key Results
The proposed FDA+STM method achieves state-of-the-art performance on several domain adaptation benchmarks for object detection:
- Clipart1k: Achieved an mAP of 41.6%, a significant improvement of +14.9% over the "Source Only" baseline.
- Comic2k: Achieved an mAP of 35.4%, outperforming existing methods by a margin of up to +8.6%.
- Watercolor2k: Achieved an mAP of 59.6%, with a substantial increase of +10.2% when STM was applied with FDA.
- Caltech Pedestrian Detection: In a challenging real-world scenario (COCO to Caltech), the method improved mAP by +5.3% to 32.4%, demonstrating its effectiveness even when FDA alone is less helpful due to high intra-domain variance.
- Ablation studies showed that both FDA and STM contribute significantly, and that Mixup regularization is crucial for stable and effective self-training.
Limitations and Future Work
- Limitations:
- The paper notes that even with their improvements, the performance on the Caltech pedestrian detection task (COCO to Caltech) is still limited, suggesting that very large domain shifts remain a challenge.
- While Mixup regularization is effective, the choice of the hyperparameter 'a' (set to 16.0) might require tuning for different datasets.
- Future Work:
- The paper implicitly suggests exploring more advanced techniques for handling extreme domain shifts.
- Further investigation into optimal Mixup strategies for object detection could be a direction.
- The authors validated their method on one-stage detectors; extending it to two-stage detectors or other architectures could be explored.
Summary updated