Abstract
Multi-Source Unsupervised Domain Adaptation (multi-source UDA) aims to learn a model from several labeled source domains while performing well on a different target domain where only unlabeled data are available at training time. To align source and target features distributions, several recent works use source and target explicit statistics matching such as features moments or class centroids. Yet, these approaches do not guarantee class conditional distributions alignment across domains. In this work, we propose a new framework called Contrastive Multi-Source Domain Adaptation (CMSDA) for multi-source UDA that addresses this limitation. Discriminative features are learned from interpolated source examples via cross entropy minimization and from target examples via consistency regularization and hard pseudo-labeling. Simultaneously, interpolated source examples are leveraged to align source class conditional distributions through an interpolated version of the supervised contrastive loss. This alignment leads to more general and transferable features which further improves the generalization on the target domain. Extensive experiments have been carried out on three standard multi-source UDA datasets where our method reports state-of-the-art results.
Introduction
Deep learning models often struggle when deployed in new environments due to "domain shift," where the training and testing data distributions differ significantly. Unsupervised Domain Adaptation (UDA) seeks to mitigate this by leveraging labeled source data and unlabeled target data. In practical scenarios, multiple labeled source domains are often available, leading to the challenge of Multi-Source UDA.
Most existing Multi-Source UDA methods focus on aligning marginal distributions or matching explicit statistics like class centroids. However, these approaches often fail to ensure that class conditional distributions are aligned—for instance, ensuring a "dog" in one domain maps to the same feature space as a "dog" in another. Centroid matching also assumes classes are unimodal, which is often not the case. The proposed Contrastive Multi-Source Domain Adaptation (CMSDA) framework robustly aligns these class conditional distributions across multiple sources while simultaneously adapting to the target domain.
Proposed Method
The CMSDA framework utilizes a unified architecture comprising a shared feature extractor $F$, a projection head $G$, and a classification head $C$. The training process is divided into two primary branches designed to handle source and target data differently.
Source Domain Alignment
To learn domain-invariant features, the model employs Supervised Contrastive Learning (SCL) on source data. SCL pulls representations of the same class together while pushing different classes apart. To enhance robustness, the authors apply MixUp, interpolating random pairs of source images and their labels. Since standard SCL requires hard labels, the authors introduce an Interpolated Supervised Contrastive Loss (ISCL) to accommodate the "soft" labels generated by MixUp. This process forces the model to learn domain-invariant features rather than memorizing domain-specific textures.
Target Domain Adaptation
For unlabeled target data, CMSDA uses a consistency regularization approach similar to FixMatch. The model generates weakly and strongly augmented versions of target images. If the prediction on the weakly augmented version exceeds a confidence threshold, it serves as a "hard pseudo-label" to train the network on the strongly augmented version. This encourages the model to produce consistent and confident predictions on the target domain.
Experimental Evaluation
The framework was evaluated on three standard Multi-Source UDA benchmarks:
- DomainNet: A massive benchmark with ~0.6 million images across 6 domains and 345 classes.
- MiniDomainNet: A subset of DomainNet with 4 domains and 126 classes (~140K images).
- Office-Home: A widely used dataset with ~15,500 images across 4 domains and 65 classes.
Results and Analysis
CMSDA achieves state-of-the-art performance across all three datasets, with an average accuracy of 50.42% on DomainNet (+1.72% over previous SOTA), 61.90% on MiniDomainNet, and 76.60% on Office-Home.
Ablation studies demonstrate that the gains from source-side contrastive loss (ISCL) and target-side consistency regularization are largely independent and additive. For example, adding ISCL provided significant boosts on challenging domains, such as a +2.48% accuracy increase on DomainNet's "Quickdraw" target compared to using standard cross-entropy alone. Furthermore, CMSDA effectively prevents "negative transfer" on difficult target domains where other models often underperform compared to source-only training.
Conclusion
CMSDA introduces a robust approach to Multi-Source UDA by combining interpolated supervised contrastive learning with consistency regularization. By aligning class conditional distributions, the model creates highly transferable features that generalize well to unseen target domains. Future work may explore applying contrastive losses to pseudo-labeled target data and investigating more advanced domain generalization augmentations.
Summary updated