Abstract
Multi-Source Domain Adaptation (MSDA) aims to transfer knowledge from multiple labeled source domains to an unlabeled target domain. Existing methods typically employ either joint alignment, which treats all sources as a single entity, or separate alignment, which aligns each source to the target individually. This paper introduces the Mutual Learning based Alignment Network (MLAN), a framework that integrates both strategies through a mutual learning module. By leveraging categorical and logits-based mutual learning, MLAN allows joint and separate branches to collaboratively refine predictions. Experimental results on benchmarks like DomainNet and Office-31 demonstrate that MLAN achieves state-of-the-art performance, particularly on complex datasets with significant domain gaps.
Introduction
In machine learning, the assumption that training and testing data share the same distribution often fails in practice. Unsupervised Domain Adaptation (UDA) addresses this by transferring knowledge from a labeled source domain to an unlabeled target domain. Multi-Source Domain Adaptation (MSDA) extends this to multiple source domains, introducing complexities such as domain gaps between sources and the target, as well as discrepancies among the sources themselves.
Current MSDA strategies generally fall into two categories:
- Joint Alignment: Aligns all source domains collectively with the target. While it captures shared information, it may lose domain-specific details.
- Separate Alignment: Aligns each source to the target individually. This preserves fine-grained information but neglects inter-source relationships.
The proposed MLAN framework seeks to harmonize these approaches, capturing both the "big picture" of joint alignment and the "fine details" of separate alignment.
Methodology: The MLAN Framework
The Mutual Learning based Alignment Network (MLAN) consists of three primary components designed to work in tandem.
1. Joint Alignment Branch
This branch treats the collection of source domains as a single entity. It utilizes a shared feature extractor and classifier to identify features that are invariant across all domains, providing a consensus-based prediction.
2. Separate Alignment Branch
Comprising $N$ sub-branches (one for each source), this component aligns each individual source domain with the target. The final prediction is a weighted ensemble, where weights are determined by source-target similarity and prediction certainty.
3. Mutual Learning Module
The core innovation of MLAN is the mutual learning module, which facilitates communication between the joint and separate branches:
- Categorical Mutual Learning: For high-confidence samples where both branches agree, the model employs source-guided K-means clustering to generate pseudo-labels for additional supervision.
- Logits Mutual Learning: For low-confidence samples, the model uses mutual distillation. The more "certain" branch (lower entropy) serves as a teacher for the other on a per-sample basis.
Experimental Evaluation
MLAN was evaluated on three major MSDA benchmarks:
- DomainNet: A large-scale dataset with ~0.6 million images across 6 domains and 345 categories.
- Office-31: A standard dataset with 3 domains (Amazon, Webcam, DSLR) and 31 categories.
- Digits-five: A collection of five digit datasets (MNIST, SVHN, USPS, etc.).
Results and Discussion
MLAN achieved state-of-the-art results across all tested benchmarks. On the challenging DomainNet dataset, MLAN reached an average accuracy of 53.5%, outperforming both joint-alignment (e.g., DRT+ST) and separate-alignment (e.g., DCTN) baselines.
On Office-31, the model achieved 91.4% average accuracy, with significant gains in the "Transfer to Amazon" task. Despite the dual-branch architecture, MLAN remains computationally efficient, with only a marginal increase in parameters compared to single-strategy models. The results suggest that the mutual learning strategy effectively leverages the strengths of both alignment paradigms.
Conclusion
MLAN demonstrates that joint and separate alignment strategies are complementary rather than mutually exclusive. By integrating them through a dynamic mutual learning framework, MLAN captures a richer set of features for domain adaptation. Future work may explore handling more extreme domain shifts or scenarios with non-overlapping category sets.
Summary updated