In standard binary classification, we learn to draw a boundary between two known groups: cats and dogs, spam and ham, or healthy and diseased. But in many real-world scenarios, we only know one side of the story.
A doctor might have a list of patients confirmed to have a rare disease, but the "healthy" control group likely contains undiagnosed cases. A biologist might know a few proteins that bind to a drug, but the thousands of "non-binding" proteins are simply those that haven't been tested yet. As Fuyi Li and Yang Zhang point out in their work on the human proteome, treating "absence of evidence" as "evidence of absence" creates a fundamental bias that cripples model performance.
Positive-Unlabeled (PU) Learning is the framework designed to solve this. It allows us to train accurate classifiers using only a small set of confirmed positive examples ($P$) and a large pool of unlabeled data ($U$) that contains a hidden mixture of both positives and negatives.
The Trap of the Pseudo-Negative
The most intuitive way to handle unlabeled data is to assume it is all negative. However, this "pseudo-negative" approach is dangerous. If the unlabeled pool contains many hidden positives—as is common in fraud detection (Liang Chen) or claim verification (Dustin Wright)—the model will be penalized for correctly identifying a positive instance.
This creates a "noisy" supervision signal. As Junbo Zhao and Long Lin argue, this ambiguity makes it nearly impossible for a standard model to establish a reliable decision boundary because the unlabeled set provides contradictory signals. The model essentially tries to learn that a specific feature is both a sign of a "positive" (from the $P$ set) and a "negative" (from the $U$ set).
How PU Learning Untangles the Signal
PU learning works by acknowledging that the unlabeled set $U$ is a distribution: $$U = \pi P + (1 - \pi) N$$ where $\pi$ is the "class prior"—the proportion of hidden positives in the unlabeled pool. To build intuition, we can categorize PU methods into three main strategies:
1. Identifying Reliable Negatives
The most straightforward approach is a two-stage process. First, we identify "reliable negatives" ($RN$) from the unlabeled pool—samples that look nothing like our known positives. Stephen Wan and Cécile Paris demonstrated this in social media filtering by using language model "perplexity" to find posts that were statistically distant from the target topic. Once these reliable negatives are found, the problem reverts to standard supervised learning.
2. Risk-Consistent Loss Functions
Instead of picking specific samples, we can modify the loss function to account for the expected contamination. If we know (or can estimate) the class prior $\pi$, we can mathematically "cancel out" the effect of the hidden positives in the unlabeled set. Jessa Bekker and Jesse Davis developed methods like TIcE to estimate this prior efficiently using decision trees. By adjusting the weight of the unlabeled samples in the loss function, we ensure the model isn't unfairly punished for identifying a hidden positive.
3. Label Distribution Alignment
A more recent perspective, introduced by Yunrui Zhao and Qianqian Xu in their Dist-PU framework, shifts the focus from individual samples to the global distribution. If we know that roughly 10% of our population should be positive, we can force the model's average predictions across the entire dataset to align with that 10% target. This "alignment" acts as a powerful regularizer, preventing the model from collapsing into a state where it predicts everything as negative.
Why It Works: The Probabilistic Gap
The core reason PU learning is possible at all is the probabilistic gap. Even if we don't have negative labels, the features of the positive class usually cluster together. Fengxiang He and Dacheng Tao show that by leveraging the density ratio between the $P$ and $U$ sets, we can recover the true posterior probability of an instance being positive.
Essentially, if a certain feature appears frequently in the labeled positive set but rarely in the unlabeled set, it is a very strong indicator of positivity. Conversely, if a feature appears frequently in the unlabeled set but almost never in the labeled positive set, it is likely a signature of the negative class.
Applications Across Domains
The versatility of this framing has led to breakthroughs in diverse fields:
- Medical Imaging: Adrian Wolny and Anna Kreshuk use PU learning for instance segmentation in crowded 3D images where only a few objects are manually traced.
- Genomics: Zeev Waks uses it to identify rare cancer "driver" genes by treating the vast majority of the genome as unlabeled rather than confirmed "non-drivers."
- Cybersecurity: Jiajing Wu and Zibin Zheng apply it to detect Bitcoin mixing services, where only a few malicious actors are known, but the rest of the network is a mystery.
By moving away from the rigid "positive vs. negative" binary and embracing the "positive vs. unknown" reality, PU learning allows models to learn from the vast amounts of data that were previously considered too "noisy" or "incomplete" to be useful.