Abstract
Unsupervised Multi-Source Domain Adaptation (MSDA) often suffers from a "pseudo-label trap," where models become overconfident in easy-to-learn classes while neglecting harder ones. We introduce S³DA-LC, a framework that utilizes label-specific confidence scores to adaptively adjust pseudo-labeling thresholds. By comparing predicted target distributions against source priors and employing Tukey’s fences for robustness, S³DA-LC effectively balances the learning process. Our method achieves new state-of-the-art performance on major benchmarks, including Office-31, Office-Home, and DomainNet.
Introduction
Multi-Source Domain Adaptation (MSDA) aims to leverage labeled data from multiple source domains to improve performance on an unlabeled target domain. A popular strategy is self-supervision via pseudo-labeling, where the model treats its own predictions as ground truth. However, this creates a feedback loop: "easy-to-learn" labels receive many pseudo-labels, while "hard-to-learn" labels are ignored. This imbalance, known as the "pseudo-label trap," severely limits the model's ability to generalize across the entire dataset.
The S³DA-LC Framework
The proposed Self-Supervised multi-Source Domain Adaptation with Label-specific Confidence (S³DA-LC) framework addresses this by moving away from fixed confidence thresholds. Instead, it employs adaptive, label-specific thresholds ($\delta_c$) to balance the training signal.
Estimating Difficulty and Adaptive Thresholding
The model estimates class difficulty by comparing the average label distribution of the source domains (the "prior") against the distribution of its own ensemble predictions on the target domain.
- Enriching Hard Labels: If a class is predicted less frequently than expected, the model lowers the confidence threshold to include more samples.
- Constraining Easy Labels: If a class is over-predicted, the threshold is raised to ensure only the most certain samples are used.
To prevent outliers from skewing these thresholds, the authors employ Tukey’s fences, a statistical method for neutralizing extreme values in the distribution ratios.
Key Results and Insights
S³DA-LC consistently outperforms existing state-of-the-art (SOTA) methods across standard computer vision benchmarks:
- Office-Home: Achieved 80.7% accuracy, a significant jump over the previous SOTA (77.4%). On the difficult "Clipart" task, it improved accuracy by 4.0%.
- DomainNet: Reached 54.8% average accuracy on this large-scale 345-class challenge.
- Office-31: Surpassed previous bests with an average accuracy of 92.4%.
Ablation studies highlight that the label-specific confidence mechanism is the primary driver of success. The most important insight is that while object appearance changes across domains, the relative frequency of labels across multiple sources provides a reliable guide for the target distribution.
Conclusion
S³DA-LC provides a straightforward yet effective mechanism for overcoming the pseudo-label trap in MSDA. By actively fixing internal biases through adaptive thresholding, the model forces itself to learn features for difficult classes that would otherwise be ignored. Future work may address the assumption of a shared label space and the potential for source-domain biases to propagate to the target.
Summary updated