Abstract
Generative Adversarial Networks (GANs) are a powerful subclass of generative models. Yet, how to effectively train them to reach Nash equilibrium is a challenge. A number of experiments have indicated that one possible solution is to bound the function space of the discriminator. In practice, when optimizing the standard loss function without limiting the discriminator's output, the discriminator may suffer from lack of convergence. To be able to reach the Nash equilibrium in a faster way during training and obtain better generative data, we propose constrained generative adversarial networks, GAN-C, where a constraint on the discriminator's output is introduced. We theoretically prove that our proposed loss function shares the same Nash equilibrium as the standard one, and our experiments on mixture of Gaussians, MNIST, CIFAR-10, STL-10, FFHQ, and CAT datasets show that our loss function can better stabilize training and yield even better high-quality images.
Introduction
The fundamental goal of a Generative Adversarial Network (GAN) is to reach a Nash equilibrium—a state where the generator produces data so realistic that the discriminator can only guess with 50% probability ($D(x) = 0.5$) whether an image is real or fake. However, standard training often fails to achieve this. Without constraints, the discriminator can become "too good" too quickly, especially in the early stages of training. This leads to vanishing or exploding gradients, leaving the generator without useful feedback. While existing methods like Wasserstein GANs (WGAN) use Lipschitz constraints to limit the gradient of the discriminator, they do not explicitly guide the discriminator toward the 0.5 equilibrium point. GAN-C addresses this by directly managing the discriminator's output divergence.
Methodology: The GAN-C Framework
The core innovation of GAN-C is the introduction of a new penalty term, $h(G, D)$, added to the standard discriminator loss:
$$h(G, D) = \mathbb{E} \left[ (\log D(x_r) - \log D(G(z)))^2 \right]$$
This term calculates the squared difference between the log-probability of real samples ($x_r$) and fake samples ($G(z)$). By minimizing this difference (weighted by a hyperparameter $\lambda$), the model discourages the discriminator from becoming overly confident. This "pulls" the outputs for real and fake data closer together, preventing the discriminator from wandering into regions of the function space far from the equilibrium. Crucially, the authors mathematically prove that this modified loss function shares the exact same Nash equilibrium as the original GAN formulation, ensuring that the fundamental objective remains unchanged.
Experimental Evaluation
The researchers evaluated GAN-C against several benchmarks to prove its versatility and stability:
- 2D Mixture of Gaussians: Used to visualize mode collapse and convergence stability.
- MNIST, CIFAR-10, & STL-10: Standard datasets for evaluating Inception Score (IS) and Fréchet Inception Distance (FID).
- FFHQ & CAT: High-resolution datasets ($256 \times 256$) to test the model's ability to handle complex textures.
Results and Stability
GAN-C demonstrated significant improvements in both image quality and training stability. On CIFAR-10, GAN-C achieved an Inception Score of 8.53, outperforming SNGAN (8.22) and RGAN (8.15). On the FFHQ face dataset, it achieved a lower FID of 58.80 compared to the standard GAN's 61.61.
In 2D Gaussian experiments, GAN-C consistently converged to the correct distribution across multiple runs. As shown in the figure above, while standard GANs often suffer from mode collapse or fail to organize points into target clusters, GAN-C successfully captures all data modes even with mismatched learning rates between the generator and discriminator.
Conclusion
GAN-C provides a direct and effective method for stabilizing GAN training by constraining the discriminator's output. While the hyperparameter $\lambda$ requires careful tuning (defaulting to 0.3), the method scales effectively to high-resolution images and offers a robust path to Nash equilibrium. Future work may explore more sophisticated mathematical expressions for the constraint and its interaction with other emerging generative architectures.
Summary updated