Sample Selection for Universal Domain Adaptation

Omri Lifshitz, Lior Wolf

Abstract

Universal Domain Adaptation (UniDA) addresses the realistic scenario where source and target label sets only partially overlap. This paper proposes a sample selection strategy based on a novel transfer score that combines domain similarity and prediction certainty. By selectively applying pseudo-labeling and confidence regularization, the method effectively aligns shared classes while isolating private ones, achieving state-of-the-art performance on multiple benchmarks.

Introduction

In traditional Domain Adaptation (DA), models are typically trained under the assumption that source and target domains share an identical set of classes (Closed-set DA). However, real-world applications often involve datasets with non-overlapping categories. Universal Domain Adaptation (UniDA) represents a more flexible and challenging setting where no prior assumptions are made about the label sets. A model must successfully align shared classes, ignore source-private classes, and identify target-private samples as "unknown."

The primary challenge in UniDA is avoiding negative transfer. This occurs when a model incorrectly forces target-private samples (e.g., "Furniture") into source-shared categories (e.g., "Animals"), leading to a degradation of the entire classification system.

Methodology

The proposed approach centers on a robust Sample Selection mechanism. Instead of treating all target data uniformly, the model utilizes a scoring function to determine the most appropriate treatment for each sample.

The Scoring Scheme

The core of the method is the transfer score $s(x)$, defined as: $$s(x) = d(x) + \max \bar{y}(x)$$

  • $d(x)$ (Domain Similarity): The output from a domain discriminator. High values indicate the sample's features are similar to those in the source domain.
  • $\max \bar{y}(x)$ (Prediction Certainty): The maximum probability output from the label classifier. High values indicate the model is confident in its class assignment.

Samples with high scores are likely to belong to shared classes, whereas low-scoring samples are likely target-private.

Overview of the proposed UniDA framework showing (a) the training phase with selective losses and (b) the inference phase with threshold-based classification.

Selective Losses

To optimize the model, the authors introduce two primary selective losses:

  1. Sample Selective Pseudo-labels: High-scoring target samples are assigned pseudo-labels based on the model's own predictions. To mitigate early training errors, a dynamic threshold is employed that gradually relaxes as the model's accuracy improves.
  2. Sample Selective Confidence Regularization: For low-scoring samples identified as likely "unknowns," the model is encouraged to produce a uniform probability distribution (low confidence). This prevents the overfitting of target-private samples to source labels.

Experimental Results

The method was evaluated on four major benchmarks: Office-Home, Office31, VisDA2017, and ImageNet-Caltech. The proposed approach consistently outperformed existing state-of-the-art (SOTA) methods like UAN and USFDA. For instance, on Office-Home, the model achieved an average accuracy of 79.6%, a significant improvement over the previous SOTA of 77.0%.

Distribution of the sample transfer score s(x), demonstrating its ability to distinguish between shared and private classes compared to standard domain classification.

Analysis of the scoring components revealed that while prediction certainty is a critical signal, the combination of both domain similarity and certainty provides the most effective separation between shared and private classes, as visualized in the score distributions.

Conclusion

This paper demonstrates that a simple additive scoring scheme, combined with dynamic thresholding and selective regularization, can effectively tackle the complexities of Universal Domain Adaptation. By prioritizing "trustworthy" samples for pseudo-labeling and regularizing potential unknowns, the model achieves robust performance across diverse datasets. Future research will focus on automating the selection of the decision threshold $s_0$ to further enhance the framework's generalizability.

Summary updated

Abstract

This paper studies the problem of unsupervised domain adaption in the universal scenario, in which only some of the classes are shared between the source and target domains. We present a scoring scheme that is effective in identifying the samples of the shared classes. The score is used to select samples in the target domain for which to apply specific losses during training; pseudo-labels for high scoring samples and confidence regularization for low scoring samples. Taken together, our method is shown to outperform, by a sizeable margin, the current state of the art on the literature benchmarks.
BibTeX
@inproceedings{lifshitz2021sample,
  author = {Omri Lifshitz and Lior Wolf},
  title = {{Sample Selection for Universal Domain Adaptation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i10.17042},
}
Figure