Abstract
Multi-source Domain Adaptation (MSDA) is more practical but challenging than the conventional unsupervised domain adaptation due to the involvement of diverse multiple data sources. Two fundamental challenges of MSDA are: (i) how to deal with the diversity in the multiple source domains and (ii) how to cope with the data shift between the target domain and the source domains. In this paper, to address the first challenge, we propose a theoretical-guaranteed approach to combine domain experts locally trained on its own source domain to achieve a combined multi-source teacher that globally predicts well on the mixture of source domains. To address the second challenge, we propose to bridge the gap between the target domain and the mixture of source domains in the latent space via a generator or feature extractor. Together with bridging the gap in the latent space, we train a student to mimic the predictions of the teacher expert on both source and target examples. In addition, our approach is guaranteed with rigorous theory offered insightful justifications of how each component influences the transferring performance. Extensive experiments conducted on three benchmark datasets show that our proposed method achieves state-of-the-art performances to the best of our knowledge.
Introduction
In real-world applications, training data often originates from multiple, diverse sources rather than a single distribution. Multi-source Domain Adaptation (MSDA) aims to transfer knowledge from these labeled source domains to an unlabeled target domain. This task is complicated by source diversity—where different sources may conflict—and the domain shift between these sources and the target. STEM (Student-Teacher Ensemble Multi-source Domain Adaptation) addresses these issues through a framework that combines domain-specific experts into a robust ensemble teacher and utilizes a student model to enforce structural consistency in a shared latent space.
Methodology
STEM adopts a Student-Teacher architecture to manage the complexities of multiple data sources effectively.
The Multi-source Teacher ($h^S$)
Rather than training a single model on a pooled dataset, STEM trains individual "domain experts" for each source. These experts are aggregated into an ensemble teacher using a source domain discriminator. This discriminator identifies which source a given input most closely resembles, weighting the experts' outputs accordingly. This approach is backed by theoretical guarantees (Theorem 1), ensuring the ensemble performs at least as well as the least effective individual expert.
Joint Feature Extraction and Latent Space
A shared generator $G$ maps all domains into a latent space. Unlike traditional methods that prioritize strict alignment, STEM encourages source representations to remain somewhat distinct. This separability allows the discriminator to more accurately assign weights to the appropriate experts.
The Target-Domain Student ($h^T$)
A student model is trained to mimic the teacher's predictions across both source and target data. This student acts as a stabilizer against the potential noise of adversarial training. By applying Virtual Adversarial Training (VAT) and entropy minimization, the student enforces the "clustering assumption," ensuring that similar points in the latent space receive consistent labels. This regularization allows the student to potentially outperform the teacher on the target domain.
Experiments and Results
STEM was evaluated on three major MSDA benchmarks: Digits-five, Office-Caltech10, and DomainNet (a large-scale dataset with ~0.6 million images).
Performance Benchmarks
STEM achieved state-of-the-art (SOTA) results across all tested datasets:
- Digits-five: STEM outperformed the previous leading method, LtC-MSDA, by a margin of 3.2% on average.
- Office-Caltech10: The model reached an average accuracy of 98.2%, achieving perfect scores on specific transfer tasks.
- DomainNet: On this highly challenging benchmark, STEM surpassed the runner-up by 6.0% on average, demonstrating significant gains in difficult domains like "Quickdraw."
Conclusion
STEM provides a robust framework for MSDA by leveraging domain experts and a consistency-driven student model. The approach's success, particularly on the massive DomainNet dataset, highlights the value of maintaining source separability in latent space and the stabilizing effect of the student model. While performance is still influenced by label shift between domains, STEM's theoretical foundations and empirical results set a new standard for multi-source adaptation. Future work may explore extending this framework to open-set domain adaptation scenarios.
Summary updated