Abstract
Generative Adversarial Networks (GANs) have shown remarkable results in modeling complex distributions, but their evaluation remains an unsettled issue. Evaluations are essential for: (i) relative assessment of different models and (ii) monitoring the progress of a single model throughout training. The latter cannot be determined by simply inspecting the generator and discriminator loss curves as they behave non-intuitively. We leverage the notion of duality gap from game theory to propose a measure that addresses both (i) and (ii) at a low computational cost. Extensive experiments show the effectiveness of this measure to rank different GAN models and capture the typical GAN failure scenarios, including mode collapse and non-convergent behaviours. This evaluation metric also provides meaningful monitoring on the progression of the loss during training. It highly correlates with FID on natural image datasets, and with domain specific scores for text, sound and cosmology data where FID is not directly suitable. In particular, our proposed metric requires no labels or a pretrained classifier, making it domain agnostic.
Introduction
Training Generative Adversarial Networks (GANs) is framed as a two-player zero-sum game between a Generator ($G$) and a Discriminator ($D$). Unlike supervised learning, where a decreasing validation loss indicates progress, GAN loss curves are notoriously uninformative, often oscillating or plateauing regardless of sample quality. Current gold-standard metrics like Fréchet Inception Distance (FID) and Inception Score (IS) suffer from domain specificity and reliance on external pretrained classifiers. This research introduces the Duality Gap (DG) as a domain-agnostic metric to monitor training and rank models effectively.
Methodology: The Duality Gap
The duality gap is a concept from game theory that measures the distance of a pair of strategies from a Nash Equilibrium. For a GAN state $(G, D)$, the DG is defined as the difference between the maximum utility a discriminator could achieve against the current generator and the minimum utility the generator could achieve against the current discriminator.
The authors demonstrate that the DG provides a theoretical upper bound on the Jensen-Shannon Divergence (JSD) between real and generated distributions. Practically, the DG is estimated by training an auxiliary "adversary" network for a few steps on a held-out dataset, making it computationally efficient.
Experimental Validation
The metric was validated across diverse domains, including toy 2D mixtures (Ring, Spiral, Grid), natural images (MNIST, CIFAR-10, CelebA), and specialized fields like cosmology and audio.
Key findings include:
- High Correlation: DG correlates strongly with FID on image tasks and with domain-specific physical statistics in cosmology.
- Failure Detection: The metric successfully identifies mode collapse and non-convergence. Even when a GAN appears to converge to a single mode, a non-zero DG indicates the game is not solved.
- Model Ranking: DG ranks GAN variants (e.g., WGAN-GP vs. NS-GAN) consistently with FID without requiring pretrained models.
Conclusion
The Duality Gap addresses the "black box" nature of GAN training by providing a reliable, domain-agnostic monitoring tool. By shifting evaluation from external classifiers to the internal game-theoretic dynamics of the model, it enables more robust development across varied scientific and creative domains. Future work may involve using the Duality Gap directly as an objective function to stabilize training.
Summary updated