Abstract
CDS (Cross-Domain Self-supervised Pre-training) is a novel two-stage approach designed to enhance the generalization of deep learning models across disparate visual domains. By incorporating a targeted self-supervised pre-training step using unlabeled data from multiple target domains, CDS learns features that are simultaneously discriminative and domain-invariant. This method significantly outperforms standard ImageNet pre-training on critical tasks such as cross-domain image retrieval and universal domain adaptation.
Introduction
Deep learning models frequently encounter performance degradation when applied to data outside their training distribution, a phenomenon known as domain shift. For instance, a model trained on high-resolution photographs (the "Real" domain) often fails to accurately identify the same objects in artistic renderings or sketches.
While standard practice involves initializing models with ImageNet pre-trained weights, these features are often biased toward the specific textures and backgrounds of the source dataset. Naively applying Self-Supervised Learning (SSL) to bridge this gap can be counterproductive; models may learn to distinguish images based on their stylistic domain rather than their semantic content, effectively widening the domain gap.
Methodology
The proposed CDS framework utilizes a two-stage pipeline. Following standard supervised pre-training on ImageNet, the model undergoes Cross-Domain Self-supervision (CDS) using unlabeled data from the downstream domains of interest (e.g., Real and Painting).
CDS optimizes two primary objectives:
- In-domain Instance Discrimination: The model is trained to distinguish an image from others within the same domain. By restricting comparisons to the same style, the model is forced to prioritize semantic features over domain-specific textures.
- Cross-domain Matching (CDM): This alignment step calculates the similarity between an image in one domain and a memory bank of features from another. By minimizing the entropy of this similarity distribution, the model is encouraged to find confident matches across domains, aligning the representations of semantically identical objects regardless of their visual style.
Key Insights and Ideas
The "Naive SSL" Trap
Standard SSL methods like SimCLR or MoCo often treat semantically similar images from different domains as "negative pairs." This pushes their representations apart in the feature space, reinforcing the domain gap. CDS avoids this by explicitly modeling cross-domain relationships.
Entropy as an Aligner
Using entropy minimization for cross-domain matching provides a robust unsupervised alignment mechanism. It requires no labels, simply tasking the model with finding a confident correlate in the alternate domain. This process shifts the model's focus from texture to shape, which is more consistent across domains.
Key Results
CDS demonstrated substantial performance gains across multiple benchmarks, including CUB-200-2011, Office-Home, and Office-31:
- Unsupervised Image Retrieval: On the CUB dataset, CDS achieved a Precision@1 of 40.9%, nearly doubling the 22.6% achieved by standard ImageNet pre-training.
- Universal Domain Adaptation: Integrated with the DANCE adaptation method, CDS improved the H-score (balancing accuracy on known and unknown classes) on Office-Home from 49.2 to 70.8.
- Few-Shot Learning: In 1-shot scenarios on the CUB dataset, CDS provided a significantly stronger starting point, with accuracy jumping from baseline levels of 5-12% to over 20-28%.
Conclusion
CDS effectively bridges the domain gap by leveraging unlabeled target data to learn domain-invariant semantic features. While currently focused on domain pairs, future work may explore scaling CDS to handle dozens of domains simultaneously or integrating it with larger foundation models like CLIP or DINO to further enhance visual representation robustness.
Summary updated