Abstract
Positive and Unlabeled (PU) learning is a pervasive challenge in machine learning where only a subset of positive examples is labeled, while the remaining data is unlabeled and contains both positive and negative instances. This paper introduces LDCE (Loss Decomposition and Centroid Estimation), a novel framework that treats unlabeled data as noisy negatives. By decomposing the standard hinge loss into noise-robust and noise-sensitive components and employing a robust centroid estimator for the negative class, LDCE effectively mitigates the impact of label noise. Experimental results across synthetic, UCI benchmark, and real-world datasets demonstrate that LDCE consistently outperforms state-of-the-art PU learning methods, particularly under high noise conditions.
Introduction
In many real-world scenarios, such as fraud detection or medical diagnosis, obtaining confirmed positive instances is relatively straightforward, but identifying definitive negative cases is often costly or impossible. This scenario defines the Positive and Unlabeled (PU) learning problem. Traditional approaches typically rely on weighting unlabeled data or identifying "reliable" negatives in a two-step process. However, these methods are often sensitive to hyperparameter tuning and prone to error propagation from the initial labeling step.
The proposed LDCE method reframes the problem by acknowledging the inherent noise in the unlabeled set. Instead of individual point labeling, it focuses on the global properties of the classes—specifically the centroid of the negative distribution—to build a more robust classifier.
Methodology
The LDCE algorithm is built upon three core pillars:
- Loss Decomposition: The authors decompose the standard hinge loss into an "even" part, which is symmetric and naturally robust to label noise, and an "odd" part that captures the noise sensitivity.
- Centroid Estimation: To address the noise in the "odd" part of the loss, the algorithm estimates the centroid of the true negative class. Given that the unlabeled set is contaminated with positives, a Median-of-means estimator is used to provide a robust location for the negative center, preventing it from being skewed by hidden positive outliers.
- Optimization: The model iteratively updates the classifier weights and the estimated centroid. A covariance-based constraint is applied to ensure the estimated centroid remains within a plausible region of the feature space, enhancing generalization.
Experimental Results
The performance of LDCE was evaluated against several benchmarks, including W-SVM, Unbiased PU (UPU), and Non-negative PU (NNPU).
Synthetic and Benchmark Data
On synthetic Gaussian distributions, LDCE achieved 100% accuracy, successfully navigating the ambiguous regions where other specialized PU methods failed. On seven UCI benchmark datasets (e.g., breast cancer, banknote, mushroom), LDCE consistently ranked among the top performers. Its advantage became more pronounced as the "noise rate" ($\eta$)—the proportion of positives hidden in the unlabeled set—increased.
Real-World Applications
The method was further tested on the HockeyFight video dataset for violence detection and the USPS digit recognition dataset. In these complex scenarios, LDCE maintained superior F-measure scores compared to its competitors.
As shown in the figures above, LDCE's ability to maintain a high F-measure even at a noise rate of 0.4 highlights its robustness. For instance, in the HockeyFight dataset, LDCE achieved an F-measure of 0.783, significantly higher than L-SVM's 0.671.
Conclusion and Future Work
LDCE provides a mathematically grounded and robust framework for PU learning by focusing on loss decomposition and class centroids. By isolating the noise-sensitive components of the loss function, it avoids the pitfalls of individual point weighting. While currently optimized for linear classifiers, future work aims to extend this approach to non-linear boundaries using kernel methods. Additionally, refining the selection of the noise rate parameter through automated cross-validation remains an area for further development.
Summary updated