Generative Adversarial Networks (GANs) have revolutionized how we synthesize images, but they are notoriously finicky. Conditional GANs (cGANs)—which allow us to specify what to generate (e.g., "generate a cat")—add another layer of complexity: not only must the image look real, but it must also match the requested label.
In their NeurIPS 2019 paper, researchers from KAIST, Kakao Brain, and POSTECH introduce a simple yet powerful metric called GOLD (Gap Of Log-Densities). This metric allows cGANs to perform a "self-diagnosis," leading to better training, higher-quality output, and smarter data selection.
The Challenge of Evaluating cGANs
Training a cGAN is a balancing act. The model needs to learn two things simultaneously:
- Marginal Distribution: Does the image look like a real object?
- Conditional Distribution: Does the image actually represent the class it was assigned?
Existing methods for evaluating GANs (like the Inception Score or FID) often struggle with cGANs because they don't explicitly measure how well the image matches its specific label. Furthermore, GANs are "implicit" models—they don't provide a simple probability density function to tell us which samples are "good" or "bad." This makes it hard to know which generated samples to trust or which real samples are the most informative to label during training.
Methods: The GOLD Estimator
The core idea is the Gap Of Log-Densities (GOLD). Mathematically, it represents the difference between the log-density of the true data and the model’s generated data:
$$\text{GOLD} \approx \log p_{data}(x, c) - \log p_g(x, c)$$
The breakthrough is that this gap can be decomposed into two parts that the cGAN's discriminator already calculates:
- The Marginal Term: How "real" the image looks (derived from the standard GAN discriminator).
- The Conditional Term: How well the image matches its class (derived from the auxiliary classifier).
The authors also introduce Balanced GOLD, which scales these two terms by their standard deviations. This prevents the "realness" score from drowning out the "class accuracy" score, a common issue since learning textures is often harder than learning class boundaries.
Three Applications of GOLD
By leveraging this self-diagnostic metric, the authors propose three distinct applications to improve the cGAN pipeline:
- Example Re-weighting (Training): During training, the model focuses more on "under-estimated" samples (those with high GOLD values) to fix its weaknesses.
- Rejection Sampling (Inference): After training, the model generates many images but only "accepts" those with high GOLD scores, effectively filtering out hallucinations or misclassified images.
- Active Learning (Data Selection): When faced with a massive pile of unlabeled data, GOLD helps identify which images are the most "confusing" or "novel" to the model, telling you exactly which ones are worth paying a human to label.
Key Results
The researchers validated GOLD across a wide variety of standard image benchmarks, including simple digits (MNIST, FMNIST), real-world images (SVHN, CIFAR-10, STL-10), and large-scale scenes (LSUN).
Instead of just looking at images, the authors rigorously evaluated the models using Fitting Capacity—training a separate classifier on generated images and seeing how well it performs on real test data. The GOLD-based methods consistently outperformed baselines across all three applications:
- Training: On the SVHN dataset, GOLD-based re-weighting improved the fitting capacity from 74.43% to 76.71%.
- Inference: Using rejection sampling on CIFAR-10, GOLD improved fitting capacity from 35.15% to 35.98%, producing visibly more distinct and accurately classed images.
- Active Learning: On MNIST, using GOLD to select which samples to label reached a fitting capacity of 94.60%, significantly higher than the 92.65% achieved by random selection with the same amount of labeled data.
Limitations and Future Work
While highly effective, the approach does have a few limitations:
- Warm-up Period: GOLD relies on the discriminator being somewhat accurate. In the very early stages of training, the GOLD values might be noisy or misleading.
- Computational Overhead: Calculating the standard deviations for the "Balanced GOLD" variant adds a bit of extra computation during the training loop.
Despite these minor hurdles, "Mining GOLD" provides a mathematically grounded yet practical way to make cGANs more self-aware. Furthermore, the authors suggest that the logic of log-density gaps could eventually be applied to other areas like outlier detection or training on datasets with noisy, incorrect labels.
Summary updated