Label-Noise Robust Generative Adversarial Networks

Takuhiro Kaneko, Yoshitaka Ushiku, Tatsuya Harada

Abstract

Generative adversarial networks (GANs) are a framework that learns a generative distribution through adversarial training. Recently, their class-conditional extensions (e.g., conditional GAN (cGAN) and auxiliary classifier GAN (AC-GAN)) have attracted much attention owing to their ability to learn the disentangled representations and to improve the training stability. However, their training requires the availability of large-scale accurate class-labeled data, which are often laborious or impractical to collect in a real-world scenario. To remedy this, we propose a novel family of GANs called label-noise robust GANs (rGANs), which, by incorporating a noise transition model, can learn a clean label conditional generative distribution even when training labels are noisy. In particular, we propose two variants: rAC-GAN, which is a bridging model between AC-GAN and the label-noise robust classification model, and rcGAN, which is an extension of cGAN and solves this problem with no reliance on any classifier. In addition to providing the theoretical background, we demonstrate the effectiveness of our models through extensive experiments using diverse GAN configurations, various noise settings, and multiple evaluation metrics.

Introduction

Conditional Generative Adversarial Networks (GANs), such as cGAN and AC-GAN, have become essential tools for generating high-quality, class-specific images. By conditioning the generation process on labels, these models can learn disentangled representations, allowing users to control the output (e.g., specifying a "cat" or a "dog"). However, a significant bottleneck in training these models is the reliance on perfectly labeled datasets. In real-world scenarios, large-scale datasets are often collected via web scraping or crowdsourcing, which inevitably introduces "label noise"—where images are associated with incorrect categories.

When standard conditional GANs are trained on noisy data, they often fail to capture the true underlying distribution, resulting in blurred or mixed-class outputs. This paper introduces Label-Noise Robust GANs (rGANs), a framework designed to learn clean, class-conditional distributions even when the training labels are heavily corrupted.

Methodology: The rGAN Framework

The core innovation of rGANs is the integration of a noise transition model, represented by a matrix $T$. This matrix defines the probability $p(\tilde{y} = j | y = i)$ that a true clean label $i$ is observed as a noisy label $j$. Based on this, the authors propose two primary architectures:

  1. rAC-GAN (Robust Auxiliary Classifier GAN): This model adapts the AC-GAN by applying forward correction to its internal classifier. Instead of comparing the classifier's prediction directly to the noisy label, the prediction is first multiplied by the noise transition matrix $T$. This allows the classifier to optimize for the clean labels while being supervised by noisy ones.
  2. rcGAN (Robust Conditional GAN): This extension of the cGAN modifies the discriminator's conditioning. Rather than using the generator's target clean label directly, a noisy label is sampled based on $T$ and provided to the discriminator. This ensures that the adversarial game is played in the noisy label space of the real data, while the generator learns to map clean labels to high-quality images.

To further stabilize training under extreme noise (e.g., 90% error rates), the authors also introduce a Mutual Information regularization term.

Experimental Setup

The effectiveness of rGANs was tested across 402 different experimental conditions using the following datasets:

  • CIFAR-10 & CIFAR-100: Evaluated using both symmetric (random) and asymmetric (class-specific) noise at levels ranging from 10% to 90%.
  • Clothing1M: A real-world dataset of 1 million images with approximately 38.5% label noise, providing a benchmark for "in-the-wild" performance.

Results and Discussion

Figure 1: Comparison of generated images. (a) Clean data (unobservable). (b) Noisy training data with 50% flipped labels. (c) Baseline cGAN trained on noisy data, showing significant class confusion. (d) Proposed rcGAN trained on the same noisy data, successfully recovering clean class distributions.

Quantitative and Qualitative Performance

As illustrated in Figure 1, standard cGANs are highly sensitive to label noise. When trained on data with 50% label flips, the baseline model (c) produces ambiguous images that fail to represent the target classes. In contrast, the proposed rcGAN (d) maintains high image quality and clear class separation, closely mimicking the unobservable clean distribution (a).

Quantitatively, rGANs consistently outperformed baselines across various metrics. For instance, on CIFAR-10 with 70% symmetric noise, the Intra FID (measuring class-specific quality) for rcGAN was significantly lower than that of standard models. On the real-world Clothing1M dataset, rGANs achieved higher GAN-train scores, indicating that the generated images were more representative of the actual clothing categories.

Key Insights

  • Sensitivity Differences: Standard cGANs tend to "memorize" label noise more than AC-GANs. AC-GANs show some inherent robustness to chaotic symmetric noise, but both fail as noise levels increase.
  • Gradient De-noising: By incorporating $T$ into the loss function, rGANs effectively de-noise the training signal, allowing the model to focus on the correct features despite incorrect labels.

Conclusion and Future Work

The rGAN framework demonstrates that it is possible to train high-quality conditional generative models on imperfectly labeled data. While highly effective, the current approach assumes class-dependent noise (where noise depends only on the label). Future research may explore instance-dependent noise, where the likelihood of mislabeling depends on the specific image content. Additionally, improving the estimation of the transition matrix $T$ in extremely high-noise or high-cardinality (e.g., ImageNet) settings remains an open challenge.

Summary updated

Abstract

Generative adversarial networks (GANs) are a framework that learns a generative distribution through adversarial training. Recently, their class-conditional extensions (e.g., conditional GAN (cGAN) and auxiliary classifier GAN (AC-GAN)) have attracted much attention owing to their ability to learn the disentangled representations and to improve the training stability. However, their training requires the availability of large-scale accurate class-labeled data, which are often laborious or impractical to collect in a real-world scenario. To remedy this, we propose a novel family of GANs called label-noise robust GANs (rGANs), which, by incorporating a noise transition model, can learn a clean label conditional generative distribution even when training labels are noisy. In particular, we propose two variants: rAC-GAN, which is a bridging model between AC-GAN and the label-noise robust classification model, and rcGAN, which is an extension of cGAN and solves this problem with no reliance on any classifier. In addition to providing the theoretical background, we demonstrate the effectiveness of our models through extensive experiments using diverse GAN configurations, various noise settings, and multiple evaluation metrics (in which we tested 402 conditions in total). Our code is available at https://github.com/takuhirok/rGAN/.
BibTeX
@inproceedings{kaneko2019label,
  author = {Takuhiro Kaneko and Yoshitaka Ushiku and Tatsuya Harada},
  title = {{Label-Noise Robust Generative Adversarial Networks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=ci2gPxY3hwjr},
}
Figure