Abstract: Positive-unlabeled (PU) learning deals with the circumstances where only a portion of positive instances are labeled, while the rest and all negative instances are unlabeled, and due to this confusion, the class prior can not be directly available. Existing PU learning methods usually estimate the class prior by training a nontraditional probabilistic classifier, which is prone to give an overestimation. Moreover, these methods learn the decision boundary by optimizing the minimum margin, which is not suitable in PU learning due to its sensitivity to label noise. In this paper, we enhance PU learning methods from the above two aspects. More specifically, we first explicitly learn a transformation from unlabeled data to positive data by entropy regularized optimal transport to achieve a much more precise estimation for class prior. Then we switch to optimizing the margin distribution, rather than the minimum margin, to obtain a label noise insensitive classifier. Extensive empirical studies on both synthetic and real-world data sets demonstrate the superiority of our proposed method.
Introduction
Positive-Unlabeled (PU) learning is a critical paradigm in machine learning where the training set consists of a small set of labeled positive instances and a large set of unlabeled instances (which contain both positive and negative samples). This scenario is common in domains like medical diagnosis (where only some patients are diagnosed) or marketing (where only some customers have purchased a product).
Two fundamental challenges plague PU learning:
- Class Prior Estimation: Determining the proportion of positive samples in the unlabeled set (the class prior, $\pi$) is difficult and often prone to overestimation.
- Noise Sensitivity: Traditional classifiers like SVMs focus on the "minimum margin." In PU learning, unlabeled positives act as noise when treated as negatives, which can drastically shift the decision boundary if it relies solely on the closest points.
Prior Estimation via Optimal Transport
To address the first challenge, the authors propose PUOTMD, which utilizes entropy-regularized Optimal Transport (OT). Instead of treating prior estimation as a classification problem, it is framed as a distribution matching task.
By treating unlabeled instances as a source distribution and labeled positives as a target, the model learns a transport plan. Unlabeled points that are easily "transported" to the positive set are identified as reliable positive candidates. This geometric approach provides a significantly more accurate estimation of the class prior compared to traditional probabilistic methods.
Robust Classification via Margin Distribution
Once the prior is estimated, the focus shifts to training the classifier. Standard methods aim to maximize the minimum margin. However, this is problematic in PU learning because unlabeled positive instances are mixed with unlabeled negatives.
As shown in the figure above, a minimum-margin classifier ($h'_{min}$) is highly sensitive to "noisy" points near the boundary. PUOTMD instead optimizes the margin distribution—specifically maximizing the mean margin while minimizing its variance. This ensures that the boundary is determined by the global structure of the data rather than a few potentially mislabeled points. To further enhance performance, the method incorporates a kNN-based similarity graph, ensuring that neighboring points in the feature space are likely to share the same label.
Empirical Evaluation
The authors tested PUOTMD against state-of-the-art baselines (including nnPU) on synthetic data and eight real-world UCI datasets (e.g., Spambase, Diabetes, Banknote).
- Prior Estimation: PUOTMD consistently yielded prior estimates closer to the ground truth than competing methods like PE and CAPU.
- Classification Performance: The model achieved superior accuracy across nearly all datasets. Its robustness was particularly evident in "high-noise" scenarios where the class prior is high (e.g., $\pi = 0.7$), demonstrating the effectiveness of the margin distribution approach.
Conclusion
PUOTMD offers a robust framework for PU learning by replacing brittle components with more stable alternatives: Optimal Transport for prior estimation and Margin Distribution for classification. While it involves more complex optimization than simple neural networks, its performance gains and resistance to label noise make it a powerful tool for real-world applications where negative labels are scarce.
Summary updated