Abstract
Positive-Unlabeled (PU) learning is a challenging paradigm where models must learn from a small set of labeled positive samples and a large pool of unlabeled data containing both true positives and true negatives. Traditional PU methods often struggle with semantically poor binary labels and confirmation bias. We present LaGAM (Latent Group-Aware Meta Disambiguation), a framework that bridges the gap between PU learning and fully supervised classification. LaGAM leverages hierarchical contrastive learning to uncover latent sub-categories and a meta-learning strategy to iteratively distill true labels from the unlabeled pool. Experimental results on benchmarks and real-world medical datasets demonstrate that LaGAM significantly outperforms existing state-of-the-art methods.
Introduction
In many practical domains, such as medical diagnosis or anomaly detection, obtaining negative labels is often difficult or impossible. For instance, in Alzheimer’s disease screening, we may have confirmed cases (positives) and a vast number of undiagnosed individuals (unlabeled). This unlabeled set is inherently noisy, containing both healthy individuals (true negatives) and undiagnosed patients (false negatives).
The primary challenge in PU learning is that binary labels (Positive vs. Unlabeled) are semantically insufficient. Complex diseases like Alzheimer’s involve multiple stages (mild, moderate, severe) and diverse physiological markers. Standard PU algorithms, which often rely on simple reweighting, fail to capture this underlying structure, leading to suboptimal representations. Furthermore, models are prone to "confirmation bias," where they reinforce their own incorrect predictions on unlabeled data.
Methodology: The LaGAM Framework
LaGAM addresses these challenges through two synergistic components: Latent Group-Aware (LGA) representation learning and Meta Label Disambiguation.
1. Latent Group-Aware (LGA) Representation Learning
To enrich the semantic information available to the model, LaGAM identifies hidden sub-categories within the data using hierarchical contrastive learning. This prevents the model from learning a simple, brittle binary boundary.
As shown in Figure 2, the process involves three key steps:
- Unsupervised Clustering: K-means is used to discover latent groups in the feature space.
- Dichotomized Cut-off: To prevent the model from grouping true positives with true negatives, a "dichotomized cutoff" is applied. Two samples are only considered a "positive pair" for contrastive learning if they belong to the same cluster and share the same predicted binary label.
- Local Neighbor Smoothing: K-nearest neighbors (kNN) are used to maintain a smooth feature space, ensuring that similar samples are mapped closely together.
2. Meta Label Disambiguation
Instead of merely reweighting samples, LaGAM actively estimates the true labels of unlabeled data through a bi-level optimization (meta-learning) process. This "aggressive" distillation leads to better utilization of the unlabeled pool.
The meta-learning framework (Figure 3) operates as follows:
- Inner Loop (Virtual Train): The model performs a "virtual" training step using the current pseudo-labels assigned to the unlabeled data.
- Outer Loop: The resulting model is evaluated on a small, clean "golden" validation set. The gradients from this evaluation are backpropagated to update and refine the pseudo-labels themselves.
- Stability Mechanisms: To prevent erratic label updates, LaGAM employs Projected Gradients and an Exponential Moving Average (EMA), ensuring a gradual and stable distillation of labels.
Experimental Evaluation
The authors evaluated LaGAM across standard benchmarks (CIFAR-10, STL-10, CIFAR-100) and a real-world Alzheimer’s Disease MRI dataset.
Key Results
- State-of-the-Art Performance: LaGAM outperformed previous SOTA PU learning methods by an average of 6.8% across all benchmarks.
- Medical AI Utility: On the Alzheimer’s dataset, LaGAM achieved a 6.1% improvement, highlighting its ability to handle complex, multi-stage disease data where latent categories are naturally present but not explicitly labeled.
- Closing the Supervision Gap: Remarkably, LaGAM's performance closely approaches that of fully supervised models. On CIFAR-10, it reached 89.6% accuracy, compared to 91.3% for the supervised baseline.
Discussion and Conclusion
LaGAM demonstrates that understanding the underlying diversity of data through latent groups is crucial for effective PU learning. By moving from simple reweighting to active label distillation, the framework makes better use of the unlabeled pool while mitigating confirmation bias.
While LaGAM requires a small labeled "golden" set and involves higher computational costs due to bi-level optimization, its ability to bridge the gap to supervised learning is a significant advancement. Future work may explore reducing the dependency on the golden set and applying these principles to other weakly supervised tasks like learning from noisy labels.
Summary updated