Abstract
Cross-domain object detection (CDOD) aims to adapt detectors from a labeled source domain to an unlabeled target domain. Traditional self-training methods often suffer from a misalignment between classification confidence and localization accuracy in pseudo-labels, leading to the inclusion of noisy data or the exclusion of valuable hard examples through rigid thresholding. This paper introduces the Harmonious Teacher (HT) framework, which employs a "harmony-based" reweighting strategy and a novel learning objective to align classification and localization signals. By utilizing all pseudo-labels with soft weights, HT achieves new state-of-the-art results on major CDOD benchmarks.
Introduction: The Misalignment Problem
In the context of CDOD, models are typically trained using a "Teacher-Student" paradigm where the Teacher generates pseudo-labels on the target domain. However, a significant issue arises: the classification score is often a poor indicator of localization quality. A bounding box might receive a high classification score despite being poorly aligned with the object, or a well-aligned box might have a lower score.
Standard methods rely on hard classification thresholds (e.g., score > 0.9) to filter pseudo-labels. This approach is problematic because it either introduces "disharmonious" noisy boxes into training or discards informative "hard examples" that the model is less certain about but could learn the most from.
Methodology
The Harmonious Teacher framework addresses these issues through two primary innovations: Harmonious Model Learning and Harmonious Sample Reweighting. The implementation is primarily built upon the FCOS (fully convolutional one-stage) detector.
1. Harmonious Model Learning
To ensure the model's internal signals are consistent, the authors modify the classification branch. Instead of predicting a binary class label, the model is trained to predict the Ground-Truth IoU (GT-IoU) on the source domain. This forces the classification score to serve as a direct estimate of localization quality. On the unlabeled target domain, a self-consistency heuristic is used: the maximum IoU between a predicted box and all other predictions serves as a proxy for localization quality.
2. Harmonious Sample Reweighting
Rather than discarding low-confidence boxes, HT retains all pseudo-labels and assigns them a weight based on a Harmony Measure ($h$): $$h = p^\beta u^{(1-\beta)}$$ where $p$ is the classification score, $u$ is the predicted IoU, and $\beta$ is a balancing hyperparameter. This "soft" reweighting ensures that boxes with both high class confidence and high localization quality have the most influence, while still allowing the model to extract useful information from more difficult, lower-confidence instances.
Experimental Results
The framework was evaluated on four standard benchmarks, consistently outperforming previous state-of-the-art (SOTA) methods:
- Cityscapes $\to$ Foggy Cityscapes: Achieved 50.4% mAP, a 5% improvement over the previous best (OADA).
- Cityscapes $\to$ BDD100K: Reached 40.2% mAP, surpassing the top two-stage detector (PT) by 5.3%.
- Sim10K $\to$ Cityscapes: Achieved 65.5% AP, a significant jump from previous methods which hovered around 55-59%.
- KITTI $\to$ Cityscapes: Demonstrated robust adaptation across different sensor types and camera setups.
Ablation studies confirmed that the combination of the Harmony Measure for ranking and the Harmonious Weighting loss provides the most substantial performance gains, moving the baseline from 45.2% to 50.4% mAP on the Foggy Cityscapes task.
Conclusion
The Harmonious Teacher framework represents a shift in cross-domain object detection by moving away from binary pseudo-label filtering. By explicitly aligning classification and localization and adopting a continuous reweighting strategy, the model effectively leverages the full spectrum of generated pseudo-labels. While future work may explore its application to Transformer-based architectures like DETR, HT currently sets a new standard for robust domain adaptation in computer vision.
Summary updated