Co-supervised learning paradigm with conditional generative adversarial networks for sample-efficient classification

Hao Zhen, Yucheng Shi, Jidong J. Yang, Javad Mohammadpour VehniApplied Computing and Intelligence 2022 · DOI · OpenReview

TL;DR

This paper introduces SEC-CGAN, a novel co-supervised learning framework that uses a conditional generative adversarial network (CGAN) to generate high-quality, semantically-conditioned synthetic data. This synthetic data augments limited and imbalanced datasets, significantly improving classifier performance in an end-to-end fashion.

What Problem Does This Paper Address?

Supervised learning, especially deep learning, thrives on large amounts of labeled, class-balanced data. However, acquiring such datasets is often expensive, time-consuming, and sometimes requires expert knowledge. Furthermore, real-world datasets frequently suffer from class imbalance, where minority classes are underrepresented, leading to biased models and poor performance on critical rare events. This paper tackles the challenge of achieving sample-efficient classification on datasets that are both small in size and imbalanced in class distribution.

Methods

The core of the proposed method is the Sample-Efficient Co-supervised Learning paradigm with Conditional Generative Adversarial Networks (SEC-CGAN). This framework integrates a conditional generative adversarial network (CGAN) with a classifier in an end-to-end manner.

Here's how it works:

  • Conditional GAN (CGAN) as a Co-supervisor: A CGAN is trained alongside the classifier. The CGAN's role is to generate synthetic images that are not only realistic but also semantically conditioned (i.e., have associated labels). This allows for targeted generation of minority class samples.
  • Confidence-Aware Synthesis: The CGAN's discriminator is used not only to distinguish real from fake images but also to provide a confidence score for the generated synthetic images. This confidence score acts as a quality control mechanism, ensuring that only high-quality synthesized samples are used for classifier training. This is crucial because poorly generated synthetic data can actually harm classifier performance.
  • End-to-End Training: Unlike previous methods that relied on pre-trained GANs, SEC-CGAN trains the CGAN and the classifier jointly. This allows the CGAN to adapt its generation strategy based on the classifier's needs and the quality of the synthesized data.
  • Weighted Batch Sampling: To further address class imbalance, the method employs weighted batch sampling during training to balance the class distribution within each batch.

Datasets

The authors evaluated their SEC-CGAN on two distinct datasets:

  • Street View House Numbers (SVHN) dataset: A public dataset commonly used for digit recognition. The authors created five scenarios by subsampling the training data to 5%, 10%, 15%, 20%, and 25% to simulate small-dataset conditions.
  • Highway Image Dataset: A self-collected dataset of road weather conditions (foggy, rainy, snowy, sunny) gathered from traffic cameras across four US states. This dataset was used to test the model's performance on a real-world safety-critical application.

Key Insights and Ideas

  • CGANs as intelligent data augmenters: The paper highlights that CGANs can go beyond simple data augmentation by generating semantically meaningful and high-quality synthetic data that directly aids classifier training.
  • Discriminator as a quality gatekeeper: Using the discriminator's confidence score to filter synthetic data is a critical innovation. It prevents the model from being trained on noisy or misleading generated samples, which is a common pitfall in GAN-based augmentation.
  • End-to-end co-supervision is key: Jointly training the generator and classifier allows for a more adaptive and effective learning process, where the generator learns to produce data that is most beneficial for the classifier.
  • Addressing both data scarcity and imbalance: The SEC-CGAN framework effectively tackles two major challenges in supervised learning simultaneously, demonstrating its broad applicability.

Key Results

The SEC-CGAN consistently outperformed both a baseline ResNet-18 classifier (trained only on real data) and the state-of-the-art EC-GAN (which uses pseudo-labeling) on both datasets.

  • SVHN Dataset (5% training data): SEC-CGAN achieved 90.3% accuracy, compared to 88.6% for EC-GAN and 87.2% for the baseline.
  • Highway Image Dataset (10% training data): SEC-CGAN reached 98.3% accuracy, outperforming EC-GAN (97.8%) and the baseline (95.5%).
  • The paper also shows that as the training dataset size increases, the performance gap between SEC-CGAN and the other methods narrows, but SEC-CGAN still maintains a competitive edge.

Qualitatively, the generated images for both datasets were visually impressive, capturing the relevant semantic features (e.g., weather conditions, digit shapes) and demonstrating the quality control mechanism.

Limitations and Future Work

  • Computational Cost: Training GANs is generally computationally intensive. While the paper focuses on sample efficiency, the overall training time might still be a consideration.
  • Hyperparameter Sensitivity: The performance of GANs and the SEC-CGAN framework can be sensitive to hyperparameter choices (e.g., learning rates, weighting multiplier λ, confidence threshold β). The paper mentions parameter tuning trials for the highway dataset.
  • Generalizability: While the authors state the paradigm is generic, further exploration on a wider variety of datasets and tasks would be beneficial to fully assess its generalizability.

The authors suggest that other network architectures can be used within the SEC-CGAN paradigm to suit task-specific applications for desirable performance, implying future work could involve exploring different generator, discriminator, and classifier backbones.

Summary updated

Abstract

Classification using supervised learning requires annotating a large amount of classes-balanced data for model training and testing. This has practically limited the scope of applications with supervised learning, in particular deep learning. To address the issues associated with limited and imbalanced data, this paper introduces a sample-efficient co-supervised learning paradigm (SEC-CGAN), in which a conditional generative adversarial network (CGAN) is trained alongside the classifier and supplements semantics-conditioned, confidence-aware synthesized examples to the annotated data during the training process. In this setting, the CGAN not only serves as a co-supervisor but also provides complementary quality examples to aid the classifier training in an end-to-end fashion. Experiments demonstrate that the proposed SEC-CGAN outperforms the external classifier GAN (EC-GAN) and a baseline ResNet-18 classifier. For the comparison, all classifiers in above methods adopt the ResNet-18 architecture as the backbone. Particularly, for the Street View House Numbers dataset, using the 5% of training data, a test accuracy of 90.26% is achieved by SEC-CGAN as opposed to 88.59% by EC-GAN and 87.17% by the baseline classifier; for the highway image dataset, using the 10% of training data, a test accuracy of 98.27% is achieved by SEC-CGAN, compared to 97.84% by EC-GAN and 95.52% by the baseline classifier.
BibTeX
@inproceedings{zhen2022co,
  author = {Hao Zhen and Yucheng Shi and Jidong J. Yang and Javad Mohammadpour Vehni},
  title = {{Co-supervised learning paradigm with conditional generative adversarial networks for sample-efficient classification}},
  booktitle = {Applied Computing and Intelligence},
  year = {2022},
  doi = {10.3934/aci.2023002},
}
Figure