In many real-world machine learning scenarios, we are tasked with learning from Positive and Unlabeled (PU) data. The central challenge in PU learning is Mixture Proportion Estimation (MPE): determining the exact fraction of the unlabeled data that belongs to the positive class. While early methods assumed that positive and negative classes were easily separable, modern research focuses on the more difficult reality where class distributions overlap significantly.
The Irreducibility Trap
Most classical MPE algorithms rely on the irreducibility assumption. This mathematical constraint posits that there must be at least one region in the feature space that contains only positive examples. If this holds, an algorithm can "anchor" its estimates to these pure regions to deduce the overall mixture.
However, as Tongliang Liu and Masashi Sugiyama point out in Rethinking Class-Prior Estimation for Positive-Unlabeled Learning, real-world data frequently violates this. When the negative class is diverse enough to mimic the positive class across the entire feature space, the irreducibility assumption fails. In these cases, standard estimators suffer from a systematic positive bias, consistently overestimating the proportion of positive samples because they cannot distinguish between pure positive regions and regions where classes are inherently mixed.
Building Intuition: Why Overlap Causes Bias
Imagine trying to estimate the proportion of salt in a saltwater mixture by looking for "pure salt" crystals. If the salt is fully dissolved, you will never find a pure region. An algorithm expecting pure regions will look at the saltiest drop it can find and assume that drop represents the "pure" baseline. If even that drop is only 10% salt, the algorithm will miscalculate the entire volume's concentration.
This phenomenon is documented across various domains:
- Data Augmentation: Polina Kirichenko shows in Understanding the Detrimental Class-level Effects of Data Augmentation that aggressive augmentation can push samples from one class into the feature space of another, creating artificial overlaps that confuse classifiers.
- Model Calibration: Muthu Chidambaram demonstrates in On the Limitations of Temperature Scaling for Distributions with Overlaps that standard uncertainty fixes like temperature scaling fail mathematically when classes share support, as the model cannot recover the true posterior in ambiguous regions.
Moving Beyond Irreducibility
To solve this, researchers have developed "wrapper" techniques and meta-algorithms that relax the need for pure regions.
One prominent approach is Regrouping CPE (ReCPE), introduced by Tongliang Liu. Instead of estimating the prior directly on the messy original data, ReCPE creates an auxiliary distribution that is easier to separate, effectively "cleaning" the signal before estimation. Similarly, Yilun Zhu and Clayton Scott proposed SuMPE (Subsampling Mixture Proportion Estimation) in Mixture Proportion Estimation Beyond Irreducibility. SuMPE works by intelligently filtering or subsampling the data to satisfy new, relaxed identifiability conditions, allowing for unbiased estimates even when the target signal "hides" inside background noise.
Simultaneous Estimation and Learning
Rather than treating MPE as a pre-processing step, recent work suggests that the class prior and the classifier should be learned together. Masahiro Kato and Gang Niu argue in Alternate Estimation of a Classifier and the Class-Prior from Positive and Unlabeled Data that an iterative approach prevents the error propagation inherent in two-step methods. By alternating between refining the class prior and updating the classifier, the model can gradually find a more consistent boundary in the overlapping feature space.
This shift from "finding pure regions" to "modeling overlapping mixtures" represents the modern frontier of learning from unlabeled data, enabling robust detection in fields ranging from tax evasion to ecological monitoring.