Introduction
While Generative Adversarial Networks (GANs) excel at producing realistic images, they frequently struggle to represent "minority" groups or sparse regions within a dataset. This limitation often leads to poor diversity and mode collapse. To address this, researchers introduce Self-Diagnosing GAN (Dia-GAN), a novel approach that leverages the discriminator’s own training history to identify underrepresented samples. By emphasizing these samples through weighted sampling, Dia-GAN significantly improves both the quality and diversity of generated images.
The Problem: Majority Bias in GANs
Although GANs have become incredibly proficient at generating high-fidelity images, they exhibit a strong "majority bias." They tend to focus on the most common patterns in a dataset while ignoring rare or complex samples—such as a specific minority demographic in a face dataset or a rare class in a multi-object dataset.
This bias leads to two primary issues:
- Low Fidelity: Samples generated from minor groups often appear blurry or distorted.
- Mode Collapse: The generator may stop producing minor groups entirely, opting to "play it safe" by satisfying the discriminator with common, easily generated samples.
Existing solutions typically rely on post-processing (like rejection sampling) or pre-processing (removing "hard" samples). However, these methods usually trade away diversity to gain quality. The goal of Dia-GAN is to boost diversity without sacrificing overall image quality.
Methods: A Self-Diagnosis Mechanism
The core innovation of Dia-GAN is a "self-diagnosis" mechanism that operates without requiring extra labels or complex hybrid models. The training process is divided into three phases:
- Phase 1: Diagnose. The GAN is trained normally for an initial period. During this time, the system tracks the Log-Density-Ratio (LDR) for every real sample in the training set. Derived from the discriminator's output, the LDR indicates how "underrepresented" a sample is. Two key metrics are calculated:
- LDRM (Mean): A high average discrepancy indicates a sample resides in a region the generator is completely missing (a "dropped mode").
- LDRV (Variance): High fluctuation in discrepancy points to "hard-to-learn" minor features that the generator struggles to stabilize.
- Phase 2: Emphasize. Training continues, but data is now sampled based on a Discrepancy Score: $s(x) = LDRM + k\sqrt{LDRV}$. Samples that the generator is failing to reproduce are shown to the model more frequently.
- Phase 3: Correct. Because weighted sampling inherently biases the model toward rare samples, Discriminator Rejection Sampling (DRS) is applied at the very end. This step "re-balances" the output distribution so it matches the original data proportions.
Key Insights
- The Discriminator as a Diagnostic Tool: Typically, only the discriminator's final state is valued. This research demonstrates that the history of the discriminator's struggle against the generator acts as a map of exactly what the model is failing to learn.
- The Importance of Variance: While previous works focused on mean discrepancy, Dia-GAN highlights that LDRV (Variance) is a powerful signal for identifying minor semantic features (e.g., glasses or hats in face datasets) that the model hasn't mastered.
- Real vs. Fake Weighting: Unlike prior methods that re-weight generated samples, Dia-GAN re-weights real training samples. If a mode is already missing, re-weighting fake samples cannot bring it back; the model must be forced to examine the real examples it is ignoring.
Experimental Results
The authors validated Dia-GAN across several benchmarks, including toy datasets (a mixture of 25 Gaussians and Colored MNIST), standard benchmarks (CIFAR-10 and CelebA), and high-resolution datasets (FFHQ at 256x256 with StyleGAN2).
- Significant FID Improvements: On CIFAR-10, Dia-GAN improved the Fréchet Inception Distance (FID) from 26.90 (Vanilla SNGAN) to 19.66, representing a massive jump in overall quality.
- Diversity Boost: On CelebA, the method significantly improved "Recall" (a measure of diversity), especially for rare attributes like "Bald," "Mustache," or "Wearing Hat." For instance, the occurrence of the "Bald" attribute in generated samples increased from 0.67% to 0.83%, moving much closer to the real-world distribution.
- Scalability: The method successfully enhanced StyleGAN2 on the FFHQ dataset, reducing FID from 14.07 to 11.89, proving its efficacy on state-of-the-art, high-resolution architectures.
Limitations and Future Work
While Dia-GAN drastically improves Recall (diversity), it can sometimes result in a slight drop in Precision (fidelity). As diversity increases, maintaining perfect quality for every rare edge case remains a challenge. Additionally, the current two-phase approach requires an initial "diagnostic" phase before weighted sampling begins. Future work could explore "online" versions where diagnosis and emphasis occur simultaneously from the start. Finally, tracking statistics for every training sample introduces minor computational and memory overhead compared to standard GAN training.
Summary updated