Wasserstein GAN With Quadratic Transport Cost

Huidong Liu, Xianfeng Gu, Dimitris Samaras

Abstract

Generative Adversarial Networks (GANs) often suffer from training instability due to the nature of their min-max optimization. While Wasserstein GANs (WGANs) improved this by using the $l_1$ transport cost, they remain limited by the 1-Lipschitz constraint. This paper introduces WGAN-QC, a framework that utilizes a quadratic ($l_2^2$) transport cost. By implementing a novel Optimal Transport Regularizer (OTR) derived from Brenier’s theorem, WGAN-QC achieves superior image quality, faster training times, and guaranteed convergence in scenarios where traditional WGANs fail.

Introduction: The $l_1$ vs. $l_2^2$ Dilemma

The primary challenge in training GANs is maintaining a stable equilibrium between the generator and the discriminator. The WGAN family addressed this by minimizing the Wasserstein distance. However, most variants rely on the $l_1$ cost, which requires the discriminator to be 1-Lipschitz. Theoretical analysis suggests that $l_1$-based WGANs can fail to converge even on simple distributions. Quadratic transport costs are theoretically more robust but have historically been difficult to implement because their dual form potentials cannot be easily unified into a single discriminator function.

Methodology: WGAN-QC

WGAN-QC overcomes these hurdles through three primary innovations:

  1. Potential Regression: The discriminator is trained to regress Kantorovich potentials. For each batch, a small Linear Programming (LP) problem is solved to determine the optimal potential values.
  2. Optimal Transport Regularizer (OTR): Based on Brenier’s theorem, the OTR enforces a physical relationship between the discriminator’s gradient and the distance between real and synthetic samples. This ensures the discriminator computes the exact quadratic Wasserstein distance.
  3. Two-Step Optimization: The training loop alternates between solving the LP for target potentials, updating the discriminator to fit these targets under the OTR constraint, and finally updating the generator.

Key Insights: Stability via Physics

Unlike heuristic penalty terms, the OTR is derived from the first-order optimality conditions of the transport problem. This grounded approach allows the discriminator more flexibility than the restrictive 1-Lipschitz constraint.

Figure 4: Convergence comparison on a toy 1D Dirac distribution. WGAN-QC (red) demonstrates rapid, stable convergence to zero, while WGAN-div (cyan) oscillates and CRGAN (blue) converges significantly slower.

As shown in the figure above, WGAN-QC provides a much more stable training signal. By computing the exact distance at every update, it avoids the approximations that lead to oscillations in other models.

Empirical Results

WGAN-QC was evaluated on benchmarks including CelebA-HQ, LSUN Bedroom, and ImageNet.

Image Quality and Fidelity

The model achieved a Fréchet Inception Distance (FID) of 7.7 on CelebA-HQ, significantly outperforming WGAN-div (13.5) and PGGAN (14.1). On the ImageNet dog subset, it reached an Inception Score of 10.48, surpassing BigGAN.

Figure 5: Qualitative comparison of generated images. Red boxes in WGAN-GP (a) and WGAN-div (b) highlight common artifacts and mode collapse issues that are mitigated in the WGAN-QC results.

Training Efficiency

Despite the added complexity of an LP solver, WGAN-QC is remarkably fast. It reached convergence on CelebA-HQ in just 0.6 days, compared to 2.1 days for WGAN-div and 5.0 days for WGAN-GP. This efficiency stems from the stability of the gradients, requiring fewer discriminator updates per generator step.

Conclusion and Future Directions

WGAN-QC represents a significant step forward in making Wasserstein GANs both theoretically sound and practically efficient. Future work includes extending the framework to conditional GANs and scaling to higher resolutions ($1024 \times 1024$). Additionally, automating the tuning of the OTR coefficient $\gamma$ remains an open area for improvement.

Summary updated

Abstract

Wasserstein GANs are increasingly used in Computer Vision applications as they are easier to train. Previous WGAN variants mainly use the lι transport cost to compute the Wasserstein distance between the real and synthetic data distributions. The lι transport cost restricts the discriminator to be 1-Lipschitz. However, WGANs with lι transport cost were recently shown to not always converge. In this paper, we propose WGAN-QC, a WGAN with quadratic transport cost. Based on the quadratic transport cost, we propose an Optimal Transport Regularizer (OTR) to stabilize the training process of WGAN-QC. We prove that the objective of the discriminator during each generator update computes the exact quadratic Wasserstein distance between real and synthetic data distributions. We also prove that WGAN-QC converges to a local equilibrium point with finite discriminator updates per generator update. We show experimentally on a Dirac distribution that WGAN-QC converges, when many of the lι cost WGANs fail to [22]. Qualitative and quantitative results on the CelebA, CelebA-HQ, LSUN and the ImageNet dog datasets show that WGAN-QC is better than state-of-art GAN methods. WGAN-QC has much faster runtime than other WGAN variants.
BibTeX
@inproceedings{liu2019wasserstein,
  author = {Huidong Liu and Xianfeng Gu and Dimitris Samaras},
  title = {{Wasserstein GAN With Quadratic Transport Cost}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00493},
}
Figure