Most: multi-source domain adaptation via optimal transport for student-teacher learning

Tuan Nguyen, Trung Le, He Zhao, Quan Hung Tran, Truyen Nguyen, Dinh Q. PhungUAI 2021 · OpenReview

Abstract

Multi-source domain adaptation (DA) is more challenging than conventional DA because the knowledge is transferred from several source domains to a target domain. To this end, we propose in this paper a new framework, MOST (Multi-Source Domain Adaptation via Optimal Transport), which tackles the challenge of transferring knowledge from multiple labeled source domains to a single unlabeled target domain by framing the problem as a "student-teacher" imitation learning task and leveraging the mathematical machinery of Optimal Transport.

Introduction

In machine learning, models often encounter domain shift, where the data they are trained on (source domain) differs significantly from real-world data (target domain). While standard Domain Adaptation (DA) deals with a single source, Multi-Source Domain Adaptation (MSDA) is a more realistic yet complex scenario where a model must combine knowledge from multiple, potentially conflicting sources to generalize to a new environment.

Existing MSDA methods frequently struggle with two primary issues: Data Shift, where raw features like lighting or backgrounds vary, and Label Shift, where the proportions of classes differ between domains. MOST addresses these by viewing MSDA as an imitation learning problem.

Methodology

The MOST framework relies on two main pillars to achieve robust knowledge transfer:

1. The Student-Teacher Paradigm

Instead of simple feature alignment, MOST treats the problem as imitation learning.

  • The Teacher: A "combined expert" that aggregates knowledge from individual source-domain classifiers. A source domain discriminator weights these experts based on their relevance to specific target samples.
  • The Student: A classifier acting on the target domain that aims to mimic the teacher’s behavior.

2. Optimal Transport (OT) for Imitation

To facilitate imitation across domains, MOST utilizes Optimal Transport. This provides a mathematical framework to "move" the target distribution toward the source distribution with minimal cost. A specific "clustering view" of OT is used to encourage target samples to find clusters of source samples, ensuring that category-specific features are aligned even under label shift.

The MOST architecture featuring source domain experts (Teacher) and a student classifier aligned via Wasserstein (WS) distance.

Experimental Evaluation

MOST was evaluated on three standard MSDA benchmarks: Digits-five, Office-Caltech10, and Office-31. The model achieved state-of-the-art (SOTA) performance across the board:

  • Digits-five: Reached an average accuracy of 95.4%, a significant jump over the previous best of 91.8%.
  • Office-Caltech10: Achieved 98.1% average accuracy, nearly saturating the benchmark.
  • Office-31: Hit 86.4% average accuracy, outperforming the runner-up by 1.8%.
Performance comparison on the Digits-five benchmark, where MOST achieves a new state-of-the-art average accuracy of 95.4%.

Discussion and Conclusion

The core insight of MOST is that domain adaptation can be viewed as an imitation game. By using Optimal Transport as a bridge, a student model can mimic a dynamically weighted ensemble of teachers, effectively neutralizing the domain gap. The clustering approach specifically mitigates label shift by ensuring that "car" features in the target domain map to "car" features in the source, regardless of class frequency.

While highly effective, future work remains to address the computational complexity of OT for high-resolution datasets and to extend the framework to "Open-set" DA, where domains do not share the exact same set of classes.

Summary updated

Abstract

Multi-source domain adaptation (DA) is more challenging than conventional DA because the knowledge is transferred from several source domains to a target domain. To this end, we propose in this pap...
BibTeX
@inproceedings{nguyen2021most,
  author = {Tuan Nguyen and Trung Le and He Zhao and Quan Hung Tran and Truyen Nguyen and Dinh Q. Phung},
  title = {{Most: multi-source domain adaptation via optimal transport for student-teacher learning}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=6n3uqn7aNg},
}
Figure