Abstract
Conditional generative adversarial networks (cGANs) have gained a considerable attention in recent years due to its class-wise controllability and superior quality for complex generation tasks. We introduce a simple yet effective approach to improving cGANs by measuring the discrepancy between the data distribution and the model distribution on given samples. The proposed measure, coined the gap of log-densities (GOLD), provides an effective self-diagnosis for cGANs while being efficiently computed from the discriminator. We propose three applications of the GOLD: example re-weighting, rejection sampling, and active learning, which improve the training, inference, and data selection of cGANs, respectively. Our experimental results demonstrate that the proposed methods outperform corresponding baselines for all three applications on different image datasets.
Introduction
Conditional Generative Adversarial Networks (cGANs) are the powerhouses behind modern image synthesis, allowing for high-quality generation tailored to specific classes. However, training them is notoriously unstable, and evaluating their performance often requires external models. A major challenge is that GANs are "implicit" models; they learn to produce samples without explicitly defining the underlying probability density of the data. This makes it difficult to diagnose exactly where a model is failing—whether it is producing low-quality images (poor marginal distribution) or clear images of the wrong class (poor conditional distribution).
The GOLD Estimator
The core innovation of this work is the GOLD (Gap Of Log-Densities) estimator. Mathematically, it represents the difference between the log-density of the real data and the model-generated data: $$\text{GOLD}(x, c) \approx \log p_{data}(x, c) - \log p_g(x, c)$$
This gap can be decomposed into two intuitive parts: a Marginal Term, which measures raw generation quality, and a Conditional Term, which measures class accuracy. Crucially, these terms can be computed almost "for free" from the two branches of a standard cGAN discriminator.
As shown in Figure 1, the GOLD estimator provides a clear diagnostic signal during training. By monitoring the gap between real and generated distributions, we can directly observe and improve the model's fitting capacity over time.
Applications of GOLD
We apply the GOLD metric in three distinct ways to enhance cGAN performance across the model lifecycle:
1. Example Re-weighting
During training, the model uses GOLD to focus more on "under-estimated" samples—those where the model's density is significantly lower than the data density. This self-correction mechanism forces the model to fix its specific mistakes, such as failing to generate a particular class accurately.
2. Rejection Sampling
During inference, GOLD acts as a quality filter. By generating a pool of images and only "accepting" those with low GOLD values, we can effectively filter out low-quality or misclassified results.
Qualitative results on the SVHN dataset (Figure 3) demonstrate that combining both marginal and conditional terms (the full GOLD metric) produces significantly cleaner and more accurate digits than random sampling or using only one component of the metric.
3. Active Learning
GOLD can also guide data selection. When deciding which real-world images to label, the system picks unlabeled images with high GOLD values. These represent the samples the model currently understands the least, maximizing the information gain from each labeled sample.
Key Results
The effectiveness of GOLD was validated using Fitting Capacity—a metric measuring how well a classifier trained on generated data performs on real data.
- Training Improvement: On the SVHN dataset, GOLD-based re-weighting improved fitting capacity from 74.33% to 76.71%.
- Inference Quality: Rejection sampling boosted performance on SVHN from 73.58% to 75.06%.
- Data Efficiency: In active learning scenarios on MNIST, GOLD-based selection reached a fitting capacity of 94.60%, outperforming random selection (92.65%) with the same amount of labeled data.
Conclusion
GOLD provides a simple yet powerful self-diagnosis tool for conditional GANs. By leveraging the discriminator's internal logic, we can improve training stability, sample quality, and data selection efficiency. Future research will explore using GOLD for outlier detection and training under noisy label conditions.
Summary updated