Introduction
Source-Free Unsupervised Domain Adaptation (SFUDA) addresses the challenge of adapting a pre-trained source model to a new, unlabeled target domain without access to the original source training data. This is particularly relevant in scenarios involving data privacy or massive datasets where sharing raw data is impractical. The core difficulty lies in "domain shift," where the source model's predictions on target data become unreliable. Existing SFUDA methods often rely on pseudo-labels, which can lead to confirmation bias—a phenomenon where the model reinforces its own incorrect predictions. This issue is exacerbated when the feature-space clusters of the target data contain samples from multiple ground truth classes, making simple cluster-based labeling risky.
Methodology: The JMDS Score
To mitigate confirmation bias and improve adaptation reliability, this research introduces the Joint Model-Data Structure (JMDS) score. This novel confidence metric combines the predictive certainty of the source model with the intrinsic geometric structure of the target data.
1. Score Components
The JMDS score is calculated by integrating two distinct perspectives:
- Model Probability of Pseudo-Label (MPPL): This "model-wise" component represents the pre-trained source model's confidence in its own prediction for a given target sample.
- Log-Probability Gap (LPG): This "data-structure-wise" component utilizes a Gaussian Mixture Model (GMM) to cluster target features. LPG measures how well a sample fits into its assigned cluster relative to the decision boundaries.
By multiplying these two probabilities, the JMDS score ensures that a sample is only considered high-confidence if both the model and the data distribution agree on its classification.
2. CoWA-JMDS and Weight Mixup
The authors propose the Confidence score Weighting Adaptation (CoWA-JMDS) framework, which uses the JMDS score as a sample-wise weight in the training loss function. This approach effectively prioritizes reliable samples during the adaptation process. Additionally, the framework introduces Weight Mixup, a technique that blends both the images and their corresponding JMDS confidence scores. This allows the model to safely incorporate lower-confidence samples into training by mixing them with high-confidence ones, facilitating a more robust exploration of the target domain's structure.
Experimental Evaluation
The method was evaluated on three major benchmarks: Office-31, Office-Home, and VisDA-2017. These tests covered closed-set, open-set, and partial-set adaptation scenarios.
Key Results
- State-of-the-Art Performance: CoWA-JMDS consistently outperformed existing methods like SHOT and NRC, achieving 90.3% accuracy on Office-31 and 86.9% on VisDA-2017.
- Superior Confidence Estimation: As shown in the Risk-Coverage curves, the JMDS score is a significantly more reliable indicator of true label correctness compared to standard metrics like Maxprob or Entropy.
- Curriculum Learning Dynamics: The distribution of JMDS scores shifts toward higher values as training progresses, creating a natural curriculum where the model gradually adapts to more difficult samples.
Conclusion
The JMDS score demonstrates that a hybrid approach—combining model-based knowledge with target data structure—is essential for effective source-free adaptation. By weighting samples and employing Weight Mixup, the CoWA-JMDS framework achieves high performance and robustness across diverse domain adaptation tasks. Future work will address the limitations of Weight Mixup in open-set scenarios to further improve the distinction between known and unknown classes.
Summary updated