Connecting Domains and Contrasting Samples: A Ladder for Domain Generalization

Tianxin Wei, Yifan Chen, Xinrui He, Jingrui HeSubmission to ICLR 2024 · Rejected

Abstract

Distribution shifts between training and testing datasets, contrary to classical machine learning assumptions, frequently occur in practice and impede model generalization performance. Studies on domain generalization (DG) thereby arise, aiming to predict the label on unseen target domain data by only using data from source domains. In the meanwhile, the contrastive learning (CL) technique, which prevails in self-supervised pre-training, can align different augmentation of samples to obtain invariant representation. It is intuitive to consider the class-separated representations learned in CL are able to improve domain generalization, while the reality is quite the opposite: people observe directly applying CL deteriorates the performance. We analyze the phenomenon with the CL theory and discover the lack of domain connectivity in the DG setting causes the deficiency. Thus we propose domain-connecting contrastive learning (\model) to enhance the conceptual connectivity across domains and obtain generalizable representations for DG. Specifically, more aggressive data augmentation and cross-domain positive samples are introduced into self-contrastive learning to improve domain connectivity. Furthermore, to better embed the unseen test domains, we propose model anchoring to exploit the domain connectivity in pre-trained representations and complement it with generative transformation loss. Extensive experiments on five standard DG benchmarks are provided. The results verify that \model~outperforms state-of-the-art baselines even without domain supervision.
BibTeX
@inproceedings{wei2025connecting,
  author = {Tianxin Wei and Yifan Chen and Xinrui He and Jingrui He},
  title = {{Connecting Domains and Contrasting Samples: A Ladder for Domain Generalization}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2025},
  doi = {10.1145/3690624.3709280},
}
Figure