TL;DR
Deep learning models often struggle when deployed in new environments due to "domain shift." This paper introduces Contrastive Multi-Source Domain Adaptation (CMSDA), a novel framework that combines interpolated supervised contrastive learning to align multiple labeled source domains, and consistency regularization to adapt to an unlabeled target domain. By effectively aligning class distributions across domains, CMSDA achieves state-of-the-art performance on several standard benchmarks.
What Problem Does This Paper Address?
When you train a model on one dataset (the source) and test it on another (the target), performance usually drops because the data distributions differ—a phenomenon known as Domain Shift. Unsupervised Domain Adaptation (UDA) tries to fix this by leveraging labeled source data and unlabeled target data. In the most practical scenarios, you might have multiple different labeled source domains (Multi-Source UDA).
Most existing Multi-Source UDA methods try to align the marginal distributions of the source and target domains, or they match explicit statistics like class centroids. However, these approaches often fail to guarantee that the class conditional distributions are aligned (i.e., making sure a "dog" in domain A maps to the same feature space as a "dog" in domain B). Centroid matching also assumes a class can be represented by a single point, which fails if the class has a multimodal distribution. This paper aims to robustly align these class conditional distributions across multiple sources while simultaneously adapting to the target domain.
Methods
The authors propose a unified architecture with a shared feature extractor, a projection head, and a classification head. The training process is split into two main branches:
-
Source Domain Alignment (MixUp + Interpolated Contrastive Learning): To learn domain-invariant features, the model uses Supervised Contrastive Learning (SCL) on the source data. SCL pulls representations of the same class together and pushes different classes apart. To make the model even more robust, the authors apply MixUp (interpolating random pairs of source images and their labels). Because MixUp creates "soft" labels, standard SCL (which requires hard labels) breaks. The authors solve this by using an Interpolated Supervised Contrastive Loss (ISCL) that can handle these mixed examples.
-
Target Domain Adaptation (Consistency Regularization): For the unlabeled target data, the model relies on semi-supervised learning techniques—specifically, a FixMatch-style approach. The model generates a "weakly augmented" and a "strongly augmented" version of a target image. If the model's prediction on the weak version is highly confident, it uses that prediction as a "hard pseudo-label" to train the network on the strongly augmented version.
Datasets
The framework is evaluated on three standard Multi-Source UDA image classification benchmarks:
- DomainNet: The largest UDA benchmark, containing ~0.6 million images across 6 distinct domains (clipart, infograph, painting, quickdraw, real, sketch) and 345 classes.
- MiniDomainNet: A smaller, less noisy subset of DomainNet featuring 4 domains and 126 classes (~140K images).
- Office-Home: A widely used dataset containing ~15,500 images across 4 domains (Art, Clipart, Product, Real-World) and 65 classes.
Key Insights and Ideas
- MixUp as Style Mixing: Applying MixUp across different source domains acts as a highly effective data augmentation strategy. It essentially "mixes" the styles of different domains, forcing the model to learn domain-invariant features rather than memorizing domain-specific textures.
- Contrastive Learning Solves Conditional Alignment: By using supervised contrastive learning on the source domains, the model naturally aligns class conditional distributions. A "car" from the Clipart domain is forced into the same latent space as a "car" from the Real-World domain, creating highly transferable features.
- Additive Gains: The ablation studies reveal that the performance gains from the source-side contrastive loss and the target-side consistency regularization are largely independent and additive.
- Preventing Negative Transfer: On highly challenging target domains (like DomainNet's "Quickdraw"), models often perform worse than if they had just trained on the source data alone (negative transfer). CMSDA successfully avoids this, maintaining robust performance even under severe domain shift.
Key Results
- State-of-the-Art Performance: CMSDA achieves the best average accuracy on all three datasets.
- DomainNet: 50.42% average accuracy (+1.72% over the previous SOTA, DAEL).
- MiniDomainNet: 61.90% average accuracy.
- Office-Home: 76.60% average accuracy.
- Ablation Impact: Adding the Interpolated Supervised Contrastive Loss (ISCL) provided significant boosts on the hardest domains, yielding up to a +2.48% accuracy jump on the DomainNet Quickdraw target and +2.75% on the Office-Home Real-World target compared to using cross-entropy alone.
Limitations and Future Work
- Hyperparameter Sensitivity: The model relies heavily on several hyperparameters, notably the contrastive temperature ($T$), the MixUp interpolation strength ($\alpha$), and the pseudo-label confidence threshold ($\tau$). The authors note that setting the temperature too high or the confidence threshold too low can introduce noisy pseudo-labels and degrade performance.
- Future Directions: Currently, the contrastive loss is only applied to the labeled source data. The authors suggest that future work could apply supervised contrastive learning to both the source data and the pseudo-labeled target data to align all conditional distributions simultaneously. They also plan to explore domain generalization augmentations like MixStyle or Fourier-based augmentations.
Summary updated