On Leveraging Unlabeled Data for Concurrent Positive-Unlabeled Classification and Robust Generation

Bing Yu, Ke Sun, He Wang, Zhouchen Lin, Zhanxing Zhu

In the world of machine learning, labeled data is gold, but unlabeled data is an ocean. Most researchers try to mine the gold; few successfully navigate the ocean. A recent paper, "On Leveraging Unlabeled Data for Concurrent Positive-Unlabeled Classification and Robust Generation," proposes a way to do both by making classification and generation work in a virtuous cycle.

TL;DR

The authors present a framework that simultaneously improves Positive-Unlabeled (PU) classification and conditional image generation. By introducing a novel Classifier-Noise-Invariant Conditional GAN (CNI-CGAN), they demonstrate that a noisy classifier can help train a high-quality generative model, which in turn generates synthetic data to further refine the classifier.

What Problem Does This Paper Address?

Deep learning models are notoriously data-hungry. While we often have a few "positive" examples of what we're looking for (e.g., images of specific car models), we rarely have a perfectly labeled dataset of everything else. Instead, we have a massive pile of unlabeled data that contains both the things we want and a lot of "negative" background noise.

This is the Positive-Unlabeled (PU) learning problem. The challenge is twofold:

  1. Classification: How do you train a model to distinguish between classes when you only know for sure what some of the positives are?
  2. Generation: How do you train a Conditional GAN (CGAN) to generate specific classes when the labels you have are scarce or provided by an imperfect (noisy) PU classifier?

Standard approaches often fail because the noise from the classifier pollutes the generative model, leading to poor image quality and "label leaking."

Methods

The researchers propose a reciprocal training framework consisting of two main pillars:

1. Multi-class PU Learning

They extend traditional binary PU learning to a $K+1$ multi-class setting (where $K$ is the number of positive classes and $+1$ represents the negative/background class). They use a non-negative PU estimator to prevent the model from overfitting to the noisy unlabeled pool, ensuring the classifier remains stable even with very few labels.

2. CNI-CGAN (Classifier-Noise-Invariant CGAN)

This is the core innovation. Instead of assuming the labels provided by the PU classifier are correct, CNI-CGAN explicitly models the label noise.

Fig. 1. Model architecture of our CNI-CGAN.
  • Confusion Matrix ($\tilde{C}$): The model estimates a matrix that represents the probability of the PU classifier mislabeling class $i$ as class $j$.
  • Noise Imitation: During training, the generator produces a sample $x_g$ for a target label $y$. This label is then "polluted" using the estimated confusion matrix to create a noisy label $\tilde{y}$.
  • Balanced Discrimination: The discriminator $(D)$ is tasked with distinguishing real pairs (Real Data, PU-predicted label) from fake pairs (Generated Data, Noise-injected label). Because both sides of the "game" incorporate the same noise mechanism, the generator is forced to learn the underlying clean distribution to fool the discriminator.

Datasets

The authors evaluated their method on three standard computer vision benchmarks, modified to fit the PU learning scenario:

  • MNIST: Digits 0-4 as positive classes.
  • Fashion-MNIST: Five non-clothing classes as positives.
  • CIFAR-10: Four transportation categories as positives.

In their experiments, they varied the "Positive Rate" (the percentage of labeled data available) to test the limits of the system, sometimes using as little as 0.2% labeled data.

Key Insights and Ideas

  • The Virtuous Cycle: The PU classifier provides (noisy) labels to train the GAN. The GAN, being noise-invariant, learns to generate high-quality, clean images. These synthetic images are then used as data augmentation to re-train and improve the PU classifier.
  • Theoretical Convergence: The authors provide a mathematical proof (Theorem 1) showing that under the assumption of disjoint sample spaces, CNI-CGAN is guaranteed to converge to the clean data distribution, even if the initial labels are highly inaccurate.
  • Robustness to "Out-of-Distribution" Data: Unlike semi-supervised learning, which assumes unlabeled data looks like labeled data, this framework handles cases where the unlabeled pool contains entirely new, unseen categories (the "negative" class).

Key Results

The framework showed significant improvements across the board:

  • Classification Boost: On MNIST with only a 0.2% positive rate, the initial PU classifier had ~69% accuracy. After augmentation from CNI-CGAN, accuracy soared to 96.33%.
  • Generative Quality: CNI-CGAN consistently achieved higher Inception Scores and Generator Label Accuracy compared to standard CGANs and other robust baselines like RCGAN-U.
Fig. 2. Generation and classification performance of CGAN-P, CGAN-A, and Ours on three datasets. Results of CGAN-P (blue lines) on PU accuracy do not exist since CGAN-P generates only K rather than K + 1 class.

Visually, the difference is striking. While baseline models often generate blurry or mislabeled images when labels are scarce, CNI-CGAN produces distinct, class-consistent samples.

Visual comparison of generated samples. CNI-CGAN produces much cleaner digits and objects compared to CGAN-A and RCGAN-U.

Limitations and Future Work

While powerful, the authors note a few areas for improvement:

  • Computational Cost: Training GANs is computationally expensive, and adding a joint optimization loop with a classifier increases this burden.
  • Scalability: The current experiments were performed on relatively low-resolution datasets. Testing the efficacy on high-resolution datasets like ImageNet remains future work.
  • Assumption of Disjoint Spaces: The theoretical proofs rely on classes having disjoint sample spaces, which might not perfectly hold in complex, real-world datasets where classes overlap significantly.

By treating classification and generation as two sides of the same coin, this research provides a robust path forward for machine learning in label-scarce environments.

Summary updated

Abstract

The scarcity of class-labeled data is a ubiquitous bottleneck in many machine learning problems. While abundant unlabeled data typically exist and provide a potential solution, it is highly challenging to exploit them. In this paper, we address this problem by leveraging Positive-Unlabeled~(PU) classification and the conditional generation with extra unlabeled data \emph{simultaneously}. We present a novel training framework to jointly target both PU classification and conditional generation when exposed to extra data, especially out-of-distribution unlabeled data, by exploring the interplay between them: 1) enhancing the performance of PU classifiers with the assistance of a novel Classifier-Noise-Invariant Conditional GAN~(CNI-CGAN) that is robust to noisy labels, 2) leveraging extra data with predicted labels from a PU classifier to help the generation. Theoretically, we prove the optimal condition of CNI-CGAN and experimentally, we conducted extensive evaluations on diverse datasets.
BibTeX
@inproceedings{yu2020leveraging,
  author = {Bing Yu and Ke Sun and He Wang and Zhouchen Lin and Zhanxing Zhu},
  title = {{On Leveraging Unlabeled Data for Concurrent Positive-Unlabeled Classification and Robust Generation}},
  booktitle = {ICIG},
  year = {2020},
  doi = {10.48550/arxiv.2006.07841},
}
Figure