Abstract
The scarcity of class-labeled data is a ubiquitous bottleneck in a wide range of machine learning problems. While abundant unlabeled data normally exist and provide a potential solution, it is extremely challenging to exploit them. In this paper, we address this problem by leveraging Positive-Unlabeled (PU) classification and conditional generation with extra unlabeled data simultaneously, both of which aim to make full use of agnostic unlabeled data to improve classification and generation performances. In particular, we present a novel training framework to jointly target both PU classification and conditional generation when exposing to extra data, especially out-of-distribution unlabeled data, by exploring the interplay between them: 1) enhancing the performance of PU classifiers with the assistance of a novel Conditional Generative Adversarial Network (CGAN) that is robust to noisy labels, 2) leveraging extra data with predicted labels from a PU classifier to help the generation. Our key contribution is a Classifier-Noise-Invariant Conditional GAN (CNI-CGAN) that can learn the clean data distribution from noisy labels predicted by a PU classifier. Theoretically, we proved the optimal condition of CNI-CGAN and experimentally, we conducted extensive evaluations on diverse datasets, verifying the simultaneous improvements on both classification and generation.
Introduction
Deep learning models are notoriously data-hungry, particularly regarding labeled examples. In many real-world scenarios, we encounter "label scarcity," where only a small subset of data is labeled as "positive," while a vast amount of unlabeled data remains available. This unlabeled data is often a complex mixture containing both positive instances and out-of-distribution background noise.
Standard semi-supervised learning typically assumes that unlabeled data follows the same distribution as labeled data. However, in Positive-Unlabeled (PU) Learning, this assumption fails. This paper introduces a reciprocal framework that simultaneously addresses two challenges: multi-class PU classification and conditional generation from noisy, limited labels.
Methodology: The Reciprocal Framework
The core of the proposed approach is a symbiotic relationship between a PU classifier and a generative model. The framework extends traditional binary PU learning to a multi-class setting ($K$ positive classes plus one negative/background class). To handle the inherent noise in unlabeled data, the authors employ a non-negative risk estimator, preventing the neural network from overfitting to incorrect labels.
Classifier-Noise-Invariant Conditional GAN (CNI-CGAN)
The primary innovation is the CNI-CGAN, designed to generate high-quality images despite receiving noisy labels from the PU classifier.
As shown in Figure 1, the architecture incorporates a confusion matrix ($\tilde{C}$) that estimates the probability of classification errors (e.g., class $i$ being mistaken for class $j$). During training, the generator "pollutes" its own labels using this matrix. This "imitation" strategy allows the discriminator to compare real images with noisy labels against fake images with similarly noisy labels, creating a robust training signal. An auxiliary loss further ensures the generator aligns with the classifier's most confident predictions, preventing class permutation.
Experimental Results
The framework was evaluated on MNIST, Fashion-MNIST, and CIFAR-10, with positive labeling rates as low as 0.2%.
As illustrated in Figure 2, the proposed method consistently outperforms baselines like CGAN-P and CGAN-A in both generator label accuracy and PU classification accuracy. Notably, on the MNIST dataset with only 0.2% labeled data, the initial PU classifier achieved approximately 69% accuracy. After the CNI-CGAN data augmentation loop, the final classification accuracy reached 96.33%. The model also demonstrated significant improvements in Inception Scores for CIFAR-10, indicating higher quality and diversity in generated images.
Conclusion
This research demonstrates that noise in unlabeled data can be modeled as a feature rather than a bug. By building a noise-invariant mechanism directly into the GAN architecture, the model can recover the true data distribution even from weak initial classifiers. While the framework shows state-of-the-art performance on standard benchmarks, future work will address the computational costs associated with scaling to larger datasets like ImageNet and investigate the model's resilience to extreme class imbalances.
Summary updated