Abstract
The existing self-supervised Multi-Source Domain Adaptation (MSDA) methods often suffer an imbalanced characteristic among the distribution of pseudo-labels. Such imbalanced characteristic results in many labels with too many or too few pseudo-labeled samples on the target domain, referred to as easy-to-learn label and hard-to-learn label, respectively. Both of these labels hurt the generalization performance on the target domain. To alleviate this problem, in this paper we propose a novel multi-source domain adaptation method, namely Self-Supervised multi-Source Domain Adaptation with Label-specific Confidence (S3DA-LC). Specifically, we estimate the label-specific confidences, i.e., the learning difficulties of labels, and adopt them to generate the pseudo-labels for target samples, enabling to simultaneously constrain and enrich the pseudo supervised signals for easy-to-learn and hard-to-learn labels. We evaluate S3DA-LC on several benchmark datasets, indicating its superior performance compared with the existing MSDA baselines.
Introduction
Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain. Multi-Source Domain Adaptation (MSDA) extends this by leveraging multiple source domains, a scenario more representative of real-world applications but significantly more complex to coordinate. A common strategy in MSDA is self-supervised pseudo-labeling, where the model generates predictions for target data to serve as training signals.
However, existing methods frequently encounter the problem of imbalanced pseudo-label distribution. As shown in Figure 1, certain classes are inherently easier for the model to recognize across domains, leading to a "rich get richer" phenomenon. These "easy-to-learn" classes dominate the pseudo-label pool with potentially noisy samples, while "hard-to-learn" classes are neglected, severely hindering the model's ability to generalize across the entire target label space.
Proposed Method: $S^3DA-LC$
To address this imbalance, we introduce Self-Supervised multi-Source Domain Adaptation with Label-specific Confidence ($S^3DA-LC$). The method operates in two distinct stages:
1. Warm-up Stage
The model first trains a shared feature extractor alongside domain-specific classifiers for each source. It calculates transferability weights for each source using Shannon entropy. If a source classifier demonstrates high certainty regarding the target data, it is assigned a higher weight in the final ensemble prediction.
2. Adaptation Stage with Label-specific Confidence
The core innovation lies in replacing fixed confidence thresholds with adaptive, label-specific thresholds ($\delta_c$).
- Estimating Difficulty: The model quantifies learning difficulty by comparing the average source label distribution (the "prior") to the predicted distribution on the target domain.
- Adaptive Thresholding: For under-represented "hard" classes, the threshold is lowered to encourage the inclusion of more samples. Conversely, for over-represented "easy" classes, the threshold is raised to filter out noise.
- Outlier Handling: We employ Tukey’s Fences to ensure that extreme distribution shifts do not result in inaccurate thresholds.
As illustrated in Figure 2, the model dynamically adjusts the confidence threshold $\delta$ based on the estimated difficulty $V/G$ for each class, effectively balancing the supervised signals.
Experimental Results
We evaluated $S^3DA-LC$ on three major benchmarks: Office-31, Office-Home, and the large-scale DomainNet. Our method consistently outperformed state-of-the-art (SOTA) baselines:
- Office-31: Achieved 92.4% average accuracy (vs. 91.6% for CAiDA).
- Office-Home: Reached 80.7% average accuracy, a significant improvement over the previous SOTA of 77.4%.
- DomainNet: On this high-diversity dataset, $S^3DA-LC$ achieved 54.8% accuracy.
Ablation studies confirmed that the label-specific confidence mechanism is the primary driver of performance. For instance, removing $\delta_c$ on the Office-Home "Clipart" task caused accuracy to drop from 70.0% to 57.5%.
Conclusion and Limitations
$S^3DA-LC$ effectively mitigates pseudo-label imbalance by using source distributions as a guide for target learning. While highly effective, the method assumes a degree of similarity between source and target label distributions and requires careful tuning of the scaling parameter $\tau$. Future work will explore extending this adaptive thresholding logic to source-free domain adaptation scenarios.
Summary updated