Introduction
Positive-Unlabeled (PU) learning—training a classifier using only positive and unlabeled data—is a crucial technique when confirmed negative examples are unavailable (e.g., diagnosing patients where the general population is unlabeled). A fundamental building block of PU learning is Class-Prior Estimation (CPE): estimating $\pi$, the proportion of true positives hidden within the unlabeled data.
Current state-of-the-art CPE methods rely heavily on a hidden theoretical requirement known as the irreducibility assumption. This assumption dictates that the positive data distribution cannot be entirely "contained" within the support of the negative data distribution. Unfortunately, in real-world scenarios, this assumption is frequently violated (e.g., the features of a healthy person might completely overlap with early disease stages). Worse, it is mathematically impossible to verify this assumption using only positive and unlabeled data. When violated, existing methods systematically overestimate the class prior, leading to poor downstream classification.
To address this, the authors introduce Regrouping CPE (ReCPE), a wrapper method that creates an auxiliary distribution to satisfy these theoretical assumptions, significantly reducing estimation bias.
Methods: Regrouping CPE (ReCPE)
Instead of attempting to estimate the prior on the original, potentially "broken" distributions, ReCPE transforms the problem into a new one where the irreducibility assumption is guaranteed to hold. ReCPE acts as a meta-method that can be applied on top of almost any existing CPE algorithm.
The algorithm follows these core steps:
- Initial Classification: Train a simple binary classifier (treating the unlabeled set as negative) to rank how "positive" each unlabeled sample appears.
- Regrouping: Take a small percentage ($p$) of the unlabeled samples that the classifier is most confident are positive and "copy" them into the positive set.
- Auxiliary Distribution: This creates a new, auxiliary positive distribution. By manually injecting these high-confidence samples, the method essentially "forces" the positive distribution to have a region distinct from the latent negative distribution.
- Base Estimation & Correction: Run any existing CPE method (like AlphaMax or KM2) on this new "regrouped" data, and use derived formulas to map the auxiliary estimate back to the original class prior.
Datasets and Experiments
The researchers evaluated ReCPE using both synthetic and real-world data:
- Synthetic Datasets: Two 10-dimensional Gaussian mixture datasets were used—one designed to satisfy the irreducibility assumption and one designed to violate it (the "reducible" case).
- Real-World Datasets: Nine benchmarks from the UCI machine learning database (including Adult, Bank, Covtype, and Magic04) were converted into PU tasks by treating specific classes as positive and others as negative/unlabeled.
Key Results and Insights
ReCPE provides a mathematically sound "safe wrapper" for existing estimators. The authors prove that if the irreducibility assumption already holds, ReCPE does not alter the result. However, if the assumption is violated, ReCPE is guaranteed to reduce the overestimation bias.
Empirical results strongly support this theory:
- Bias Reduction: On "reducible" synthetic data, ReCPE drastically reduced the estimation bias from 0.2 down to approximately 0.01.
- Broad Improvement: ReCPE was tested against seven state-of-the-art baseline methods (including AlphaMax, DEDPUL, and KM2). It improved the accuracy of all of them across the majority of the nine real-world datasets, suggesting that the irreducibility assumption is frequently violated in practice.
- Top Performer: The combination of AlphaMax with the regrouping wrapper (ReAM) achieved the lowest average absolute estimation error across all tested scenarios.
Limitations and Future Work
While highly effective, ReCPE introduces a hyper-parameter $p$ (the fraction of data to copy). The authors found $p=10\%$ worked well across all datasets, but the optimal value might vary depending on the specific domain. Additionally, in practical implementation, the authors "copy and paste" samples to create the auxiliary distribution, which is an approximation of the theoretical regrouping of probability masses. Future work will explore more sophisticated ways to generate the auxiliary samples to further refine the estimation process.
Summary updated