Promoting Semantic Connectivity: Dual Nearest Neighbors Contrastive Learning for Unsupervised Domain Generalization

Yuchen Liu, Yaoming Wang, Yabo Chen, Wenrui Dai, Chenglin Li, Junni Zou, Hongkai Xiong

Abstract

Unsupervised Domain Generalization (UDG) seeks to train models on unlabeled data from multiple source domains such that they generalize to unseen target domains. Traditional contrastive learning (CL) methods often succumb to "domain bias," clustering representations by style (e.g., sketches vs. photos) rather than semantic content. This paper introduces DN$^2$A (Dual Nearest Neighbors contrastive learning with strong Augmentation), a framework designed to promote semantic connectivity. By utilizing aggressive data augmentation and a novel "double-lock" nearest neighbor strategy, DN$^2$A forces the model to bridge the gap between disparate domains. Experimental results on PACS and DomainNet show that DN$^2$A outperforms previous state-of-the-art methods by significant margins, highlighting the importance of semantic connectivity in robust representation learning.

Introduction

The primary challenge in Domain Generalization (DG) is learning features that remain invariant across different data distributions. While supervised DG relies on labels, Unsupervised Domain Generalization (UDG) leverages unlabeled data. Standard Contrastive Learning (CL) frameworks like SimCLR are typically used for this task, but they often fail due to a lack of Semantic Connectivity ($C_s$).

In cross-domain settings, CL tends to exhibit high intra-domain connectivity (aligning objects of different classes within the same domain due to shared style) and low intra-class connectivity (failing to align the same object across different domains). The authors define $C_s$ as the ratio of intra-class similarity ($C_\beta$) to intra-domain similarity ($C_\alpha$). A higher $C_s$ indicates that the model is successfully prioritizing semantic content over domain-specific noise.

Conceptual overview of semantic connectivity and its correlation with generalization accuracy.

Methodology: The DN$^2$A Framework

To address the connectivity gap, the researchers propose DN$^2$A, which consists of two main components:

1. Strong Augmentation (SA)

To suppress intra-domain connectivity ($C_\alpha$), the model employs 14 types of "strong" augmentations. By heavily distorting textures and colors, the model is discouraged from relying on easy, domain-specific cues, forcing it to seek more fundamental semantic features.

2. Building Bridges with Dual Nearest Neighbors (DN$^2$)

To boost intra-class connectivity ($C_\beta$), the framework identifies positive pairs across domains using a "double-lock" mechanism:

  • Cross Domain Double-lock NN (CD$^2$NN): To avoid false matches across domains, a cross-domain neighbor is only accepted if it is a neighbor to both the original image and the image's closest in-domain neighbor. This "in-domain mediator" acts as a sanity check.
  • In-domain Cycle NN (ICNN): This strategy uses trusted cross-domain neighbors to "cycle back" and discover diverse in-domain neighbors, accounting for variations in pose or lighting within a single domain.

Experimental Results

The DN$^2$A framework was evaluated on the PACS (Photo, Art, Cartoon, Sketch) and DomainNet benchmarks.

Performance Metrics

On the PACS dataset with a 1% label fraction, DN$^2$A achieved an accuracy of 55.82%, a dramatic improvement over SimCLR's 25.93% and the previous state-of-the-art, BrAD (43.81%). Similar gains were observed on DomainNet, where DN$^2$A reached 40.56% accuracy compared to SimCLR’s 16.55%.

Visual Analysis

Visualizations provide compelling evidence of the method's efficacy. t-SNE plots show that while SimCLR clusters data primarily by domain, DN$^2$A successfully entangles domains, clustering by semantic category. Grad-CAM heatmaps further reveal that DN$^2$A focuses on the core semantic components of objects (e.g., the body of an elephant) rather than background or stylistic elements.

t-SNE and Grad-CAM visualizations comparing SimCLR and DN$^2$A.

Discussion and Conclusion

The success of DN$^2$A underscores the critical role of semantic connectivity in UDG. Notably, models pretrained with DN$^2$A on a fraction of the data used for ImageNet outperformed standard ImageNet pretraining, suggesting that domain-invariant features are inherently more robust. While the method requires a "warm-up" period to ensure reliable neighbor searches, its ability to align features across vast domain gaps without labels represents a significant step forward for unsupervised learning. Future work may extend this connectivity-based approach to multi-modal learning environments.

Summary updated

Abstract

Domain Generalization (DG) has achieved great success in generalizing knowledge from source domains to unseen target domains. However, current DG methods rely heavily on labeled source data, which are usually costly and unavailable. Since unlabeled data are far more accessible, we study a more practical unsupervised domain generalization (UDG) problem. Learning invariant visual representation from different views, i.e., contrastive learning, promises well semantic features for in-domain unsupervised learning. However, it fails in cross-domain scenarios. In this paper, we first delve into the failure of vanilla contrastive learning and point out that semantic connectivity is the key to UDG. Specifically, suppressing the intra-domain connectiv-ity and encouraging the intra-class connectivity help to learn the domain-invariant semantic information. Then, we propose a novel unsupervised domain generalization approach, namely Dual Nearest Neighbors contrastive learning with strong Augmentation (DN 2 A). Our DN 2 A leverages strong augmentations to suppress the intra-domain connectivity and proposes a novel dual nearest neighbors search strategy to find trustworthy cross domain neighbors along with in-domain neighbors to encourage the intra-class connectivity. Experimental results demonstrate that our DN 2 A outperforms the state-of-the-art by a large margin, e.g., 12.01% and 13.11 % accuracy gain with only 1% labels for linear evaluation on PACS and DomainNet, respectively.
BibTeX
@inproceedings{liu2023promoting,
  author = {Yuchen Liu and Yaoming Wang and Yabo Chen and Wenrui Dai and Chenglin Li and Junni Zou and Hongkai Xiong},
  title = {{Promoting Semantic Connectivity: Dual Nearest Neighbors Contrastive Learning for Unsupervised Domain Generalization}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.00342},
}

Loading related content...

Figure