Abstract
Multi-source domain adaptation (DA) is more challenging than conventional DA because the knowledge is transferred from several source domains to a target domain. To this end, we propose in this paper a new framework, MOST (Multi-Source Domain Adaptation via Optimal Transport), which tackles the challenge of transferring knowledge from multiple labeled source domains to a single unlabeled target domain by framing the problem as a "student-teacher" imitation learning task and leveraging the mathematical machinery of Optimal Transport.
Introduction
In machine learning, models often encounter domain shift, where the data they are trained on (source domain) differs significantly from real-world data (target domain). While standard Domain Adaptation (DA) deals with a single source, Multi-Source Domain Adaptation (MSDA) is a more realistic yet complex scenario where a model must combine knowledge from multiple, potentially conflicting sources to generalize to a new environment.
Existing MSDA methods frequently struggle with two primary issues: Data Shift, where raw features like lighting or backgrounds vary, and Label Shift, where the proportions of classes differ between domains. MOST addresses these by viewing MSDA as an imitation learning problem.
Methodology
The MOST framework relies on two main pillars to achieve robust knowledge transfer:
1. The Student-Teacher Paradigm
Instead of simple feature alignment, MOST treats the problem as imitation learning.
- The Teacher: A "combined expert" that aggregates knowledge from individual source-domain classifiers. A source domain discriminator weights these experts based on their relevance to specific target samples.
- The Student: A classifier acting on the target domain that aims to mimic the teacher’s behavior.
2. Optimal Transport (OT) for Imitation
To facilitate imitation across domains, MOST utilizes Optimal Transport. This provides a mathematical framework to "move" the target distribution toward the source distribution with minimal cost. A specific "clustering view" of OT is used to encourage target samples to find clusters of source samples, ensuring that category-specific features are aligned even under label shift.
Experimental Evaluation
MOST was evaluated on three standard MSDA benchmarks: Digits-five, Office-Caltech10, and Office-31. The model achieved state-of-the-art (SOTA) performance across the board:
- Digits-five: Reached an average accuracy of 95.4%, a significant jump over the previous best of 91.8%.
- Office-Caltech10: Achieved 98.1% average accuracy, nearly saturating the benchmark.
- Office-31: Hit 86.4% average accuracy, outperforming the runner-up by 1.8%.
Discussion and Conclusion
The core insight of MOST is that domain adaptation can be viewed as an imitation game. By using Optimal Transport as a bridge, a student model can mimic a dynamically weighted ensemble of teachers, effectively neutralizing the domain gap. The clustering approach specifically mitigates label shift by ensuring that "car" features in the target domain map to "car" features in the source, regardless of class frequency.
While highly effective, future work remains to address the computational complexity of OT for high-resolution datasets and to extend the framework to "Open-set" DA, where domains do not share the exact same set of classes.
Summary updated