Abstract
Generative Adversarial Networks (GANs) often suffer from training instability due to the nature of their min-max optimization. While Wasserstein GANs (WGANs) improved this by using the $l_1$ transport cost, they remain limited by the 1-Lipschitz constraint. This paper introduces WGAN-QC, a framework that utilizes a quadratic ($l_2^2$) transport cost. By implementing a novel Optimal Transport Regularizer (OTR) derived from Brenier’s theorem, WGAN-QC achieves superior image quality, faster training times, and guaranteed convergence in scenarios where traditional WGANs fail.
Introduction: The $l_1$ vs. $l_2^2$ Dilemma
The primary challenge in training GANs is maintaining a stable equilibrium between the generator and the discriminator. The WGAN family addressed this by minimizing the Wasserstein distance. However, most variants rely on the $l_1$ cost, which requires the discriminator to be 1-Lipschitz. Theoretical analysis suggests that $l_1$-based WGANs can fail to converge even on simple distributions. Quadratic transport costs are theoretically more robust but have historically been difficult to implement because their dual form potentials cannot be easily unified into a single discriminator function.
Methodology: WGAN-QC
WGAN-QC overcomes these hurdles through three primary innovations:
- Potential Regression: The discriminator is trained to regress Kantorovich potentials. For each batch, a small Linear Programming (LP) problem is solved to determine the optimal potential values.
- Optimal Transport Regularizer (OTR): Based on Brenier’s theorem, the OTR enforces a physical relationship between the discriminator’s gradient and the distance between real and synthetic samples. This ensures the discriminator computes the exact quadratic Wasserstein distance.
- Two-Step Optimization: The training loop alternates between solving the LP for target potentials, updating the discriminator to fit these targets under the OTR constraint, and finally updating the generator.
Key Insights: Stability via Physics
Unlike heuristic penalty terms, the OTR is derived from the first-order optimality conditions of the transport problem. This grounded approach allows the discriminator more flexibility than the restrictive 1-Lipschitz constraint.
As shown in the figure above, WGAN-QC provides a much more stable training signal. By computing the exact distance at every update, it avoids the approximations that lead to oscillations in other models.
Empirical Results
WGAN-QC was evaluated on benchmarks including CelebA-HQ, LSUN Bedroom, and ImageNet.
Image Quality and Fidelity
The model achieved a Fréchet Inception Distance (FID) of 7.7 on CelebA-HQ, significantly outperforming WGAN-div (13.5) and PGGAN (14.1). On the ImageNet dog subset, it reached an Inception Score of 10.48, surpassing BigGAN.
Training Efficiency
Despite the added complexity of an LP solver, WGAN-QC is remarkably fast. It reached convergence on CelebA-HQ in just 0.6 days, compared to 2.1 days for WGAN-div and 5.0 days for WGAN-GP. This efficiency stems from the stability of the gradients, requiring fewer discriminator updates per generator step.
Conclusion and Future Directions
WGAN-QC represents a significant step forward in making Wasserstein GANs both theoretically sound and practically efficient. Future work includes extending the framework to conditional GANs and scaling to higher resolutions ($1024 \times 1024$). Additionally, automating the tuning of the OTR coefficient $\gamma$ remains an open area for improvement.
Summary updated