Abstract
Heterogeneous unsupervised domain adaptation (HUDA) is the most challenging domain adaptation setting where the feature space differs between source and target domains, and the target domain has only unlabeled data. Existing HUDA methods assume that both positive and negative examples are available in the source domain, which may not be satisfied in some real applications. This paper addresses a new challenging setting called positive and unlabeled heterogeneous domain adaptation (PU-HDA), a HUDA setting where the source domain only has positives. PU-HDA can also be viewed as an extension of PU learning where the positive and unlabeled examples are sampled from different domains. A naive combination of existing HUDA and PU learning methods is ineffective in PU-HDA due to the gap in label distribution between the source and target domains. To overcome this issue, we propose a novel method, positive-adversarial domain adaptation (PADA), which can predict likely positive examples from the unlabeled target data and simultaneously align the feature spaces to reduce the distribution divergence between the whole source data and the likely positive target data. PADA achieves this by a unified adversarial training framework for learning a classifier to predict positive examples and a feature transformer to transform the target feature space to that of the source. Specifically, they are both trained to fool a common discriminator that determines whether the likely positive examples are from the target or source domain. We experimentally show that PADA outperforms several baseline methods, such as the naive combination of HUDA and PU learning.
Introduction: The Challenge of PU-HUDA
In the field of transfer learning, Unsupervised Domain Adaptation (UDA) is a standard task where a model is trained using a labeled "source" dataset for an unlabeled "target" dataset. However, standard UDA assumes the source and target share the same features. Heterogeneous UDA (HUDA) increases complexity by assuming different feature spaces (e.g., text descriptions vs. numerical metadata).
This paper identifies a further, more difficult gap: PU-HUDA. In many real-world scenarios, the source domain only contains "positive" labels. For instance, a jewelry store (source) may only know who did buy jewelry (positives), while a partner bank (target) has a database of unlabeled customers with entirely different features (account balances vs. purchase history).
Standard HUDA methods fail in this setting because they attempt to align the entire target distribution with the positive-only source distribution. As illustrated in the figure above, this "smears" the target's positive and negative classes together, destroying the model's predictive power.
The PADA Framework
To overcome the label distribution gap, the authors propose PADA (Predictive Adversarial Domain Adaptation), which consists of three interacting components:
- Feature Transformer ($F$): Maps the target's unique features into the source's feature space.
- Classifier ($C$): Predicts which target samples are likely "positives."
- Discriminator ($D$): Acts as a judge, trying to distinguish between actual source positives and the "likely positives" from the target domain.
In this adversarial loop, $F$ and $C$ work together to "fool" $D$. $C$ identifies target samples it predicts as positive, and $F$ transforms them so they look indistinguishable from source positives to the Discriminator. To prevent model collapse during early training, the authors introduce Soft-Labeling ($PADA_S$). A base classifier ($C_0$) is trained on "common features" shared by both domains (e.g., basic demographics) to provide probabilistic hints that guide the main classifier until the adversarial process stabilizes.
Experimental Evaluation
Since PU-HUDA is a new task, the authors reorganized three existing datasets to simulate this environment:
- Movielens-Netflix: Movie ratings where genres serve as features.
- 20-Newsgroups: Text classification where different categories are mapped to source and target domains.
- Default of Credit Card Clients: Credit records where the "positive ratio" in the target data was varied from 0.5 down to 0.1 to test robustness.
Key Insights and Results
- Label Distribution Gap: Standard HUDA fails in PU settings because it assumes a balanced source. If the source is 100% positive, a standard alignment algorithm will try to make the entire target look like a positive class.
- Asymmetric Transformation: It is more effective to only transform target features toward the source space, reducing optimization complexity.
- Common Features as Anchors: Shared features act as a vital anchor. Using them to create a "teacher" model ($C_0$) helps the adversarial network learn faster and avoid trivial solutions.
PADA and its soft-labeled variant, $PADA_S$, consistently outperformed baselines. On the Movielens-Netflix dataset, PADA improved accuracy by approximately 4-6% over standard PU-learning and HUDA combinations. In the credit card dataset, even with only 10% actual positives in the target data, $PADA_S$ maintained high performance, whereas other methods saw significant drops in AUC.
Conclusion
PADA successfully aligns disparate feature spaces while accurately identifying positive samples in unlabeled data. While the current model assumes common features do not have massive distribution divergence, future work aims to incorporate techniques to reduce divergence specifically within these shared features to make the soft-labeling anchor even more reliable.
Summary updated