TL;DR
Standard domain generalization (DG) relies on the assumption that every class appears across multiple training domains. This paper introduces the problem of domain-linked classes—classes that only appear in a single source domain during training—and proposes FOND, a contrastive and fairness-regularized framework that transfers domain-invariance learned from shared classes to domain-linked ones.
What Problem Does This Paper Address?
In real-world settings (such as medical imaging across hospitals or autonomous driving in distinct geographies), certain rare categories or anomalies are only observed in one environment. Standard DG techniques assume classes are domain-shared ($\mathcal{Y}_S$), allowing models to isolate domain-invariant features by comparing the same class across multiple domains.
When a category is domain-linked ($\mathcal{Y}_L$), however, models easily latch onto spurious correlations between the domain's style and the class label. At test time in an unseen target domain, performance on these domain-linked categories collapses. The authors study how to transfer knowledge of domain variations from shared classes to linked classes so models can generalize out-of-distribution even when a class was only ever seen in one domain.
Methods
The authors introduce FOND (Fair and cONtrastive Domain-linked learning), which optimizes a composite loss: $$\mathcal{L}_{\text{FOND}} = \mathcal{L}_{\text{task}} + \lambda_{\text{xdom}} \mathcal{L}_{\text{xdom}} + \lambda_{\text{fair}} \mathcal{L}_{\text{fair}}$$
Key components include:
- Domain-Aware Contrastive Loss ($\mathcal{L}_{\text{xdom}}$): Instead of standard contrastive learning, FOND applies two reweighting factors:
- Inter-domain positive boosting ($\alpha \ge 1$): Amplifies similarity pull between positive pairs from different domains, encouraging representation invariance.
- Intra-domain negative boosting ($\beta \ge 1$): Amplifies repelling forces between samples of different classes from the same domain. This acts as hard negative mining, discouraging the network from using shared domain features as classification shortcuts.
- Fairness Regularization ($\mathcal{L}_{\text{fair}}$): Because domain-invariant contrastive incentives can disproportionately favor domain-shared classes, FOND enforces a fairness constraint between class groups: $$\mathcal{L}_{\text{fair}} = |\mathcal{L}_{\text{task}}^{(L)} - \mathcal{L}_{\text{task}}^{(S)}|$$ This penalizes the performance gap between domain-linked ($\mathcal{Y}_L$) and domain-shared ($\mathcal{Y}_S$) classes, pushing the representation to generalize equally well across both sets.
Datasets
The authors adapted three standard DomainBed benchmark datasets into domain-linked evaluation protocols:
- PACS (4 domains: Photo, Art-painting, Cartoon, Sketch; 7 classes; 9,991 images; style shifts)
- VLCS (4 domains: VOC2007, LabelMe, Caltech101, SUN09; 5 classes; 10,729 images; photographic/real-world shifts)
- OfficeHome (4 domains: Art, Clipart, Product, Real-World; 65 classes; 15,588 images; style shifts)
Experiments evaluate two setups:
- Low-shared: $\sim 1/3$ of classes are domain-shared, $\sim 2/3$ are domain-linked.
- High-shared: $\sim 2/3$ of classes are domain-shared, $\sim 1/3$ are domain-linked.
Key Insights and Ideas
- Domain-invariance transfers across classes: Even if a specific object was only observed in sketches, learning what makes other objects invariant across photos, paintings, and sketches helps the model generalize the unseen sketch-only class to target domains.
- Intra-domain negatives combat spurious features: Pushing apart different classes that share the same visual style is essential for breaking the correlation between class semantics and domain environment.
- Fairness as a generalizability equalizer: Classic DG fairness equalizes outcomes across protected demographic attributes. Here, treating domain-linked classes as a disadvantaged group forces the model to distribute invariant representations equitably rather than prioritizing shared classes.
- Absolute class counts matter: Generalization gains were strongest when the absolute number of domain-shared categories was large enough to expose the network to diverse environmental transformations.
Key Results
- Outperforming Baselines in High-Shared Settings: Across 10 competitive DG baselines (including ERM, CORAL, CausIRL, CAD, and PGrad), FOND achieved top performance on domain-linked classes:
- Average accuracy on $\mathcal{Y}_L$: Reached 43.9%, a +9.3% absolute gain (26.9% relative improvement) over the strongest baseline, ERM (34.6%).
- VLCS: Achieved 72.1% on $\mathcal{Y}_L$ compared to ERM’s 51.8% (a 39.2% relative gain).
- PACS and OfficeHome: FOND reached 19.1% (vs. ERM 14.7%) and 40.6% (vs. ERM 37.5%), respectively.
- Latent Space Analysis: t-SNE visualizations confirm that while ERM clusters target-domain representations by domain artifacts, FOND clusters domain-linked samples primarily by class semantics.
Limitations and Future Work
- Dependency on Shared Class Quantity: In the Low-shared setting, FOND's advantage drops to parity with existing baselines because there are too few shared classes to extract reliable domain-invariant representations.
- Binary Assumption: The framework assumes a strict binary split between classes (either strictly linked to 1 domain or shared across $>1$ domains). Future extensions will need to accommodate continuous or imbalanced distributions of class availability across sources.
Summary updated