Abstract
Source-free Domain Adaptation (SFDA) has emerged as a practical alternative to Unsupervised Domain Adaptation (UDA) when source data is inaccessible due to privacy or storage constraints. While various heuristic methods have shown success in SFDA, a rigorous theoretical understanding has been lacking. This paper bridges this gap by connecting SFDA heuristics—specifically discriminability and diversity—to the theory of self-training under the expansion assumption. Based on these insights, we introduce an improved SFDA framework featuring an auto-adjusting diversity constraint and a teacher-student model for learnable data augmentation. Our method achieves state-of-the-art performance on major benchmarks, demonstrating the power of theoretically-grounded algorithm design.
Introduction
Domain shift remains a significant challenge in deploying deep learning models. While Unsupervised Domain Adaptation (UDA) effectively mitigates this by aligning source and target distributions, it requires simultaneous access to both datasets. In many real-world scenarios, source data is unavailable, necessitating Source-free Domain Adaptation (SFDA). SFDA relies solely on a pre-trained source model and unlabeled target data. Despite the empirical success of "tricks" like entropy minimization and clustering, their theoretical justification remained elusive until now.
Theoretical Framework
The authors demonstrate that SFDA is a specialized form of self-training. By invoking the "expansion assumption"—which posits that the data distribution has sufficient continuity—they show that enforcing consistency across different views of the data can bound the target error. In this context, the common SFDA objectives of discriminability (confidence) and diversity (class balance) serve as proxies for the theoretical requirements of self-training. Specifically, while UDA bounds error using the distance between source and target distributions, SFDA uses diversity as a constraint to prevent the model from collapsing into a single-class solution.
Methodology
The proposed framework introduces two key innovations derived from the theoretical analysis:
- Auto-Adjusting Diversity Constraint: Unlike traditional methods that use a fixed weight for the diversity loss, this approach dynamically decays the weight as the model's discriminability improves. This reflects the theoretical insight that the diversity "nudge" is most critical during the early stages of adaptation to prevent class collapse.
- Teacher-Student Augmentation Training: Theory suggests that the target error bound is sensitive to augmentation strength. The authors implement a framework where a student model learns to classify increasingly difficult augmented images, while a teacher model (an exponential moving average of the student) ensures that these augmentations preserve the original semantic content.
Experimental Evaluation
The method was evaluated on three standard benchmarks: Office-31, Office-Home, and VisDA2017.
- Office-Home: The approach achieved a state-of-the-art average accuracy of 73.4%, a +0.7% improvement over strong baselines.
- VisDA2017: On this large-scale synthetic-to-real challenge, the method reached 88.4% per-class accuracy, the highest among compared methods.
- Generalizability: Ablation studies confirmed that the auto-adjusting mechanism consistently outperforms fixed-weight baselines. Furthermore, applying these modules to other existing SFDA methods (like SHOT-IM) yielded accuracy boosts of up to +2.9%.
Conclusion
By providing a theoretical foundation for SFDA through the lens of self-training, this work moves beyond empirical heuristics. The resulting improvements in diversity handling and data augmentation lead to robust performance gains across diverse domains. A primary remaining challenge is the sensitivity to the initial quality of the source model; if the pre-trained model starts with an extremely high error rate on the target domain, the expansion-based self-training may fail to converge. Future research will focus on handling such poorly initialized scenarios.
Summary updated