Domain Generalization for Domain-Linked Classes

Kimathi Kaai, Saad Hossain, Sirisha Rambhatla

Abstract

Domain generalization (DG) focuses on transferring domain-invariant knowledge from multiple source domains (available at train time) to an, a priori, unseen target domain(s). This requires a class to be expressed in multiple domains for the learning algorithm to break the spurious correlations between domain and class. However, in the real-world, classes may often be domain-linked, i.e. expressed only in a specific domain, which leads to extremely poor generalization performance for these classes. In this work, we aim to learn generalizable representations for these domain-linked classes by transferring domain-invariant knowledge from classes expressed in multiple source domains (domain-shared classes). To this end, we introduce this task to the community and propose a Fair and cONtrastive feature-space regularization algorithm for Domain-linked DG, FOND. Rigorous and reproducible experiments with baselines across popular DG tasks demonstrate our method and its variants' ability to accomplish state-of-the-art DG results for domain-linked classes.

The Problem: Domain-Linked Classes

Most machine learning models struggle when test data comes from a different distribution than training data (the "Out-of-Distribution" problem). Standard Domain Generalization (DG) aims to learn features that are invariant across different training "domains" (e.g., different lighting, camera types, or artistic styles).

However, existing DG research usually assumes that every class (e.g., "dog") is represented in every training domain. In reality, data is often fragmented: a medical database might only have certain rare diseases recorded at one specific hospital. These are domain-linked classes ($\mathcal{Y}_L$). Because the model only sees these classes in one context, it accidentally "links" the class features with the domain features, leading to a failure when recognizing that class in a new environment.

Conceptual illustration of domain-shared ($\mathcal{Y}_S$) and domain-linked ($\mathcal{Y}_L$) classes across source domains ($S_1, S_2, S_3$) and target domain ($T$).

Methodology: The FOND Algorithm

The authors propose FOND (Fair and cONtrastive Domain-linked learning), which rests on two main pillars:

  1. Domain-Aware Contrastive Regularization ($L_{xdom}$): FOND refines contrastive learning by specifically weighting pairs. It increases the "reward" for recognizing the same class across different domains (inter-domain positives) and increases the "penalty" for confusing different classes within the same domain (intra-domain negatives). This forces the feature extractor to ignore "style" and focus on "substance."

  2. Fair Representation Learning ($L_{fair}$): The discrepancy between domain-shared classes ($\mathcal{Y}_S$) and domain-linked classes ($\mathcal{Y}_L$) is treated as a fairness problem. A loss term minimizes the absolute difference between the classification errors of these two groups, preventing the model from over-optimizing for the "easy" shared classes at the expense of the "hard" linked classes.

Key Results

FOND consistently outperformed state-of-the-art DG baselines (like CORAL, MLDG, and SagNet), particularly in settings with a high number of shared classes.

Accuracy comparison on domain-shared vs. domain-linked classes. FOND (cyan diamond) shows superior performance on linked classes while maintaining high shared-class accuracy.
  • VLCS: Achieved a massive +20.3% improvement in accuracy for domain-linked classes compared to the best baseline.
  • PACS: Showed a +2.9% improvement.
  • OfficeHome: Showed a +1.1% improvement.

Insights and Conclusion

The core innovation of FOND is viewing domain-linked classes as an underrepresented group. By enforcing "loss parity" (fairness) between shared and linked classes, the model is forced to apply the domain-invariant "tricks" it learned from the shared classes to the linked ones. While the method struggles when very few classes are shared across domains (the "Low" setting), it provides a robust framework for tackling real-world data scarcity where domain-class correlations are prevalent.

Summary updated

Abstract

Domain generalization (DG) focuses on transferring domain-invariant knowledge from multiple source domains (available at train time) to an, a priori, unseen target domain(s). This requires a class to be expressed in multiple domains for the learning algorithm to break the spurious correlations between domain and class. However, in the real-world, classes may often be domain-linked, i.e. expressed only in a specific domain, which leads to extremely poor generalization performance for these classes. In this work, we aim to learn generalizable representations for these domain-linked classes by transferring domain-invariant knowledge from classes expressed in multiple source domains (domain-shared classes). To this end, we introduce this task to the community and propose a Fair and cONtrastive feature-space regularization algorithm for Domain-linked DG, FOND. Rigorous and reproducible experiments with baselines across popular DG tasks demonstrate our method and its variants' ability to accomplish state-of-the-art DG results for domain-linked classes. We also provide practical insights on data conditions that increase domain-linked class generalizability to tackle real-world data scarcity.
BibTeX
@misc{kaai2023domain,
  author = {Kimathi Kaai and Saad Hossain and Sirisha Rambhatla},
  title = {{Domain Generalization for Domain-Linked Classes}},
  year = {2023},
  eprint = {2306.00879},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2306.00879},
}
Figure