Inferential Wasserstein Generative Adversarial Networks

Xiao Wang, Yao Chen, Qingyi Gao, Xiao Wang 0045arXiv 2021 · arXiv

Abstract

Generative Adversarial Networks (GANs) have achieved remarkable success in synthetic data generation but continue to face challenges regarding training stability, mode collapse, and the lack of interpretable convergence metrics. This paper presents the Inferential Wasserstein Generative Adversarial Network (iWGAN), a framework that integrates auto-encoders with the Wasserstein GAN (WGAN) through a principled primal-dual optimization approach. By simultaneously learning an encoder and a generator, iWGAN provides a mathematically grounded "duality gap" as a stopping criterion and enables individual sample quality assessments, significantly mitigating mode collapse in the process.

Introduction

Generative Adversarial Networks (GANs) are a cornerstone of modern generative modeling, yet they are notoriously difficult to train. Two persistent issues plague the architecture: the tendency for the generator to "collapse" to a limited subset of the data distribution (mode collapse) and the absence of a reliable objective function to signal convergence. While the introduction of the Wasserstein distance (WGAN) improved stability by providing a smoother gradient, it did not fully resolve the mode collapse problem or provide a clear, scalar metric for training completion.

In contrast, Variational Auto-encoders (VAEs) offer stable training and meaningful latent representations but often produce blurry samples due to the limitations of their reconstruction-based objectives. The Inferential Wasserstein GAN (iWGAN) seeks to fuse these two paradigms. By leveraging the primal-dual relationship of the 1-Wasserstein distance, iWGAN establishes a framework that maintains the high-fidelity output of GANs while gaining the stability and inferential capabilities of auto-encoders.

Methodology

Primal-Dual Optimization

The core innovation of iWGAN is the simultaneous optimization of an encoder ($Q$) and a generator ($G$). This is motivated by the dual formulation of the 1-Wasserstein distance. In this setup, the encoder $Q$ maps observed data to a latent space (the primal side), while a discriminator $f$ (or critic) distinguishes between distributions (the dual side).

The iWGAN Objective

The iWGAN objective function is designed to minimize the difference between the data distribution and the model distribution while ensuring the latent space remains well-structured. The objective combines:

  1. Reconstruction Error: Ensuring that $G(Q(X))$ accurately recovers the input $X$.
  2. Distributional Matching: Using the discriminator to align the generated distribution with the target data distribution.
  3. Regularization: To maintain the Lipschitz constraint required by the Wasserstein distance, a Gradient Penalty is applied to the discriminator. Additionally, a Maximum Mean Discrepancy (MMD) penalty is utilized to ensure the encoded latent variables follow a prior distribution, such as a standard normal distribution.

The Duality Gap as a Convergence Metric

Perhaps the most significant practical contribution of iWGAN is the use of the "duality gap." By calculating the difference between the primal and dual objective values during training, the framework provides a natural, scalar metric for convergence. As the model approaches the true data distribution, the duality gap approaches zero, offering a quantifiable stopping criterion that has been historically absent in GAN research.

Experimental Setup

The authors evaluated iWGAN against several benchmarks, including WGAN-GP, ALI, and WAE, across synthetic and high-dimensional datasets:

  • Synthetic Mixtures of Gaussians: Three configurations (Ring, Swiss Roll, and a 25-mode Grid) were used to specifically test the model's resilience to mode collapse.
  • MNIST: A standard benchmark for handwritten digit synthesis and latent space consistency.
  • CelebA: A complex dataset of over 200,000 celebrity faces used to evaluate the model's ability to scale to high-dimensional, realistic image generation.

Key Results and Discussion

Mitigation of Mode Collapse

In the synthetic "Grid" experiment, which features 25 distinct Gaussian modes, standard WGAN-GP models often failed to capture the full diversity, dropping several modes. In contrast, iWGAN successfully recovered all 25 modes, demonstrating that the inclusion of the inferential encoder $Q$ forces the generator to cover the entire data manifold.

Stability and Quality

iWGAN demonstrated faster and more stable convergence compared to CycleGAN and WGAN-GP. On the CelebA dataset, the model produced sharper and more realistic facial images than previous hybrid models like ALI. Furthermore, the latent space proved to be highly structured; smooth interpolations between latent vectors resulted in smooth transitions between facial features, indicating that the model learned a continuous and meaningful representation of the data.

Individual Sample Quality Checks

Because iWGAN includes an explicit encoder, it allows for the calculation of reconstruction errors for individual samples. This provides a unique "quality score" for each generated image: lower reconstruction errors correlate with higher realism as perceived by the model, allowing for automated filtering of synthetic outputs.

Conclusion and Future Work

The Inferential Wasserstein GAN provides a theoretically rigorous bridge between auto-encoding and adversarial generation. By utilizing the duality gap, it transforms GAN training from a qualitative exercise into a quantifiable optimization process. While the current model assumes a point mass for the conditional distribution of latent variables, future work may explore more flexible inference models and the scaling of iWGAN to even larger architectures like BigGAN. Ultimately, iWGAN represents a significant step toward more stable, interpretable, and diverse generative modeling.

Summary updated

BibTeX
@misc{wang2021inferential,
  author = {Xiao Wang and Yao Chen and Qingyi Gao and Xiao Wang},
  title = {{Inferential Wasserstein Generative Adversarial Networks}},
  year = {2021},
  eprint = {2109.05652},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.05652},
}
Figure