Abstract
Unsupervised domain adaptation (UDA) aims to learn models for a target domain of unlabeled data by transferring knowledge from a labeled source domain. Traditional UDA assumes labeled source data are available. Due to privacy concerns, source-free UDA (White-Box UDA) has emerged, assuming only a trained source model is available. However, source models may be proprietary or kept as black boxes for security. This work introduces Black-Box Unsupervised Domain Adaptation (B$^2$UDA), where only the API of the source model is accessible. We propose Iterative Learning with Noisy Labels (IterLNL), which treats black-box predictions as noisy labels and iteratively refines a target model using noise-rate estimation and category-wise sampling. Experiments demonstrate that IterLNL performs comparably to traditional UDA methods despite having significantly less information.
Introduction: The Black-Box Constraint
In modern machine learning, "Model-as-a-Service" is a common paradigm. Users often interact with high-performing models via APIs without access to internal weights or the original training data. This creates a barrier for standard Unsupervised Domain Adaptation (UDA) techniques that rely on feature alignment or gradient-based fine-tuning.
As shown in Figure 1, B$^2$UDA is the most restrictive yet realistic setting. The challenge lies in adapting to a target domain when the source model is a "black box" that only provides output labels or probabilities.
Methodology: Iterative Learning with Noisy Labels (IterLNL)
The core insight of this work is to reframe domain adaptation as a Learning with Noisy Labels (LNL) problem. Since the source model is not perfectly accurate on the target domain, its predictions serve as "noisy labels." IterLNL addresses this through three key components:
- Noise Rate Estimation: To train effectively, the algorithm must estimate the proportion of incorrect labels. Because deep models are often overconfident, a rescale curve is used to map raw confidence scores to more accurate noise estimates.
- Category-wise Sampling: Domain shift affects classes differently. To prevent the model from ignoring difficult classes, IterLNL employs a category-wise sampling strategy using a probability queue buffer, ensuring balanced learning across all labels.
- Iterative Strategy: The process is iterative. A target model trained on filtered noisy labels often surpasses the source API's accuracy. This improved model then generates better labels for the next round of training, effectively "bootstrapping" its way to high performance.
Experimental Evaluation
The authors evaluated IterLNL on benchmark datasets including Office-31, VisDA-2017, and Digits.
- VisDA-2017: IterLNL achieved 80.1% accuracy, a significant improvement over the source model's initial 49.6%. It outperformed previous black-box methods like sMDA by nearly 27%.
- Office-31: The method reached an average accuracy of 87.9%, rivaling top-tier white-box methods (e.g., SHOT) that require full access to model weights.
Conclusion
IterLNL demonstrates that high-quality domain adaptation is possible even when the source model is entirely opaque. By treating domain shift as structured label noise and using iterative refinement, the method bridges the gap between black-box accessibility and white-box performance. This has significant implications for privacy-preserving AI and the practical deployment of commercial models in new environments.
Summary updated