Gradient-Based Sample Selection for Black-Box Universal Domain Adaptation

Qiuyan He, Minghua DengAAAI 2025 · DOI

Abstract

Researchers from Peking University have introduced GSS (Gradient-based Sample Selection), a novel framework designed for Black-Box Universal Domain Adaptation (B²-UniDA). In B²-UniDA, models must adapt to target domains with differing category sets while only having API-level access to a source model. GSS overcomes the limitations of traditional threshold-based methods by deriving a selection criterion from the mathematical relationship between open-set and closed-set classifiers. By utilizing gradient information and a multi-category equilibrium module, GSS achieves state-of-the-art performance across major benchmarks without requiring manual threshold tuning.

The Challenge of Black-Box Universal Domain Adaptation

Unsupervised Domain Adaptation (UDA) typically involves adapting a model from a labeled source domain to an unlabeled target domain. However, real-world applications often face two critical constraints:

  1. Category Shift (UniDA): The source and target domains may not share the same label space, containing private classes unique to each domain.
  2. Privacy and Security (Black-Box): Access to source data or internal model weights is often restricted. Practitioners only have a "Black-Box" interface where they can query the model and receive predictions.

The primary difficulty in B²-UniDA is distinguishing between "known" classes (present in the source) and "unknown" classes (novel to the target) using only prediction outputs. Most existing methods rely on manually tuned probability thresholds, which lack robustness across different datasets and scenarios.

The GSS Framework

The GSS framework addresses these challenges by looking at the intrinsic learning dynamics of the adaptation process rather than static output probabilities.

Overview of the GSS framework, illustrating the interaction between the black-box source model and the target training module.

1. Gradient-Based Sample Selection Criterion

The core innovation is the selection metric $T$, derived using Bayes’ Theorem and an analysis of how a loss function’s gradient behaves during training. Unlike arbitrary thresholds, $T$ provides a natural decision boundary:

  • If $T(x) > 0$, the sample is likely a "known" class.
  • If $T(x) < 0$, the sample is likely "unknown."

This approach allows the model to automatically separate samples based on whether the gradient "wants" to decrease the probability of the sample being categorized as unknown.

Density distribution of the selection metric T in ODA and OPDA scenarios, showing clear separation between known (red) and unknown (blue) samples.

2. Multi-category Equilibrium

To handle the inherent class imbalance between the broad "unknown" category and specific "known" classes, GSS employs a balancing mechanism. It dynamically estimates the domain adaptation type (e.g., partial-set vs. open-set) and adjusts training weights to prevent the model from collapsing into a state where it predicts all samples as the majority class.

3. Self-Training and Clustering

The model is refined through a combination of:

  • Knowledge Distillation: Transferring discriminative power from the black-box source model.
  • Self-Supervised Clustering: Using learnable prototypes to ensure that target features are grouped into cohesive, class-specific clusters, which improves the reliability of the $T$ metric.

Experimental Evaluation

The authors evaluated GSS on four standard benchmarks: Office, OfficeHome, VisDA, and DomainNet. The method was tested across various shifts, including Open-Partial Domain Adaptation (OPDA), where both domains have unique private classes.

Key Results

  • State-of-the-Art Performance: GSS consistently outperformed the previous leading black-box method, UB2DA.
  • Significant Gains on Large Datasets: On the VisDA benchmark (OPDA setting), GSS achieved an H-score of 43.5%, a massive improvement over UB2DA’s 25.5%.
  • Threshold-Free Robustness: The gradient-based criterion proved effective across diverse datasets without the need for the extensive grid-searching required by probability-based methods.
  • Efficiency: Despite the added complexity of clustering, GSS proved that a black-box interface provides sufficient signal for high-performance adaptation if processed through the lens of learning dynamics.

Conclusion and Future Work

GSS demonstrates that the gradient of the loss function contains intrinsic information about out-of-distribution samples, enabling robust black-box adaptation without manual intervention. While the method still requires a predefined number of clusters ($K_{clu}$), it represents a significant step toward automated, privacy-preserving domain adaptation. Future research may focus on automatically determining the optimal number of target clusters and improving performance when the initial domain shift is exceptionally large.

Summary updated

BibTeX
@inproceedings{he2025gradient,
  author = {Qiuyan He and Minghua Deng},
  title = {{Gradient-Based Sample Selection for Black-Box Universal Domain Adaptation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33881},
}
Figure