TL;DR
This paper introduces a new machine learning challenge: PU-HUDA (Positive and Unlabeled Heterogeneous Unsupervised Domain Adaptation). It addresses scenarios where you want to train a model for a new "target" domain using a "source" domain that has entirely different features and—crucially—only contains positive examples. The authors propose PADA (Predictive Adversarial Domain Adaptation), a framework that uses adversarial training and a "soft-labeling" mechanism to align disparate feature spaces while accurately identifying positive samples in unlabeled data.
What Problem Does This Paper Address?
In the world of transfer learning, Unsupervised Domain Adaptation (UDA) is a common task: you have a labeled "source" dataset and want to train a model for an unlabeled "target" dataset. However, standard UDA assumes the source and target share the same features (e.g., both are images of the same resolution).
Heterogeneous UDA (HUDA) ups the ante by assuming the feature spaces are different (e.g., the source uses text descriptions while the target uses numerical metadata). This paper identifies a further, more difficult gap: PU-HUDA. In many real-world cases, the source domain only has "positive" labels.
Imagine a jewelry store (source) partnering with a bank (target) to find potential customers. The jewelry store only knows who did buy jewelry (positives); it doesn't have a list of people who definitely won't. Meanwhile, the bank has a massive database of unlabeled customers, and its data features (account balances) are totally different from the store's features (past purchase history). Naively combining existing methods fails because they try to align the entire target distribution with the positive-only source distribution, effectively "smearing" the target's positive and negative classes together and destroying the model's predictive power.
Methods
The authors propose PADA (Predictive Adversarial Domain Adaptation), which consists of three interacting components:
- Feature Transformer ($F$): Maps the target's unique features into the source's feature space.
- Classifier ($C$): Predicts which target samples are likely "positives."
- Discriminator ($D$): Acts as a judge, trying to distinguish between actual source positives and the "likely positives" from the target domain.
The Adversarial Loop: $F$ and $C$ work together to "fool" $D$. $C$ identifies target samples it thinks are positive, and $F$ transforms them so they look indistinguishable from source positives to the Discriminator.
Soft-Labeling ($PADA_S$): To prevent the model from collapsing during the early stages of training (when the transformer is still "clumsy"), the authors introduce a base classifier ($C_0$). This base model is trained only on "common features"—attributes shared by both domains, like basic demographics. $C_0$ provides "soft labels" (probabilistic hints) to guide the main classifier $C$ until the adversarial process stabilizes.
Datasets
Since PU-HUDA is a new task, the authors reorganized three existing datasets to simulate this environment:
- Movielens-Netflix: Users' movie ratings where genres serve as features. One domain is treated as source (positives only), the other as target (unlabeled).
- 20-Newsgroups: A text classification dataset where different news categories are mapped to source and target domains using tf-idf features.
- Default of Credit Card Clients: A dataset of credit records. The authors varied the "positive ratio" (the percentage of actual positives in the unlabeled target data) from 0.5 down to 0.1 to test the model's robustness.
Key Insights and Ideas
- The Label Distribution Gap: The biggest "aha" moment is the realization that standard HUDA methods fail in PU settings because they assume the source has a balanced mix of labels. If the source is 100% positive, a standard alignment algorithm will try to make the entire target look like a positive class, which is counter-productive for classification.
- Asymmetric Transformation: The authors found it more effective to only transform the target features toward the source space, rather than trying to map both to a new "latent" space. This reduces the complexity of the optimization problem.
- Common Features as a "Warm-up": Even a small number of shared features (like age or gender) can act as a vital anchor. By using these to create a "teacher" model ($C_0$), the complex adversarial network learns much faster and avoids trivial solutions (like predicting everyone is negative).
Key Results
PADA and its soft-labeled variant, $PADA_S$, consistently outperformed baselines across all datasets:
- Accuracy Boost: On the Movielens-Netflix dataset, PADA improved accuracy by about 4-6% over standard PU-learning and HUDA combinations.
- Robustness to Imbalance: In the credit card dataset, even when only 10% of the target data was actually positive, $PADA_S$ maintained high performance, whereas other methods saw significant drops in AUC (Area Under the Curve).
- Feature Alignment: Qualitative analysis showed that PADA successfully brought the "likely positive" target samples closer to the source data in the feature space while keeping the "likely negative" target samples far away.
Limitations and Future Work
The authors acknowledge that their current model assumes that the "common features" themselves don't have a massive distribution divergence between domains. If the shared features (like "income") are measured differently or have vastly different scales between the bank and the jewelry store, the "soft-labeling" anchor might fail.
Future work aims to incorporate techniques to reduce the divergence specifically within these common features to make the "teacher" model even more reliable.
Summary updated