MR-GAN: Manifold Regularized Generative Adversarial Networks

Qunwei Li, Bhavya Kailkhura, Rushil Anirudh, Yi Zhou, Yingbin Liang, Pramod K. VarshneyarXiv 2018 · OpenReview

Abstract

Despite the growing interest in generative adversarial networks (GANs), training GANs remains a challenging problem, both from a theoretical and a practical standpoint. To address this challenge, in this paper, we propose a novel way to exploit the unique geometry of the real data, especially the manifold information. More specifically, we design a method to regularize GAN training by adding an additional regularization term referred to as manifold regularizer. The manifold regularizer forces the generator to respect the unique geometry of the real data manifold and generate high quality data. Furthermore, we theoretically prove that the addition of this regularization term in any class of GANs including DCGAN and Wasserstein GAN leads to improved performance in terms of generalization, existence of equilibrium, and stability. Preliminary experiments show that the proposed manifold regularization helps in avoiding mode collapse and leads to stable training.

Introduction

Generative Adversarial Networks (GANs) are the gold standard for creating synthetic data, but they are notoriously temperamental. Researchers typically face three major hurdles: Mode Collapse, where the generator only produces a limited variety of samples; Lack of Equilibrium, where the competition between the generator and discriminator fails to stabilize; and Instability, leading to oscillating training and poor-quality results.

The authors identify a fundamental geometric issue: real-world data usually lives on a low-dimensional "manifold" embedded in a high-dimensional space. When the generator's output and the real data lie on disjoint manifolds, the gradients provided by the discriminator become unhelpful or "noisy." MR-GAN addresses this by providing the generator with a geometric map of the data manifold.

Methodology

The core innovation is the Manifold Regularizer. Instead of just relying on the discriminator's feedback, the authors add a penalty term to the generator's objective function: $\lambda \|\nabla_{\mathcal{M}}(\psi(y) - \psi(x))\|^2$. This term measures how much the generated data ($y$) deviates from the intrinsic geometry of the real data ($x$).

To implement this, the model uses an embedding function ($\psi$)—often a pre-trained autoencoder—to map high-dimensional data into a lower-dimensional latent space. Since the "true" manifold is unknown, the paper uses a graph Laplacian approach to approximate the manifold's shape during training. Unlike many previous works that regularize the discriminator, MR-GAN focuses on the generator, ensuring it receives stable gradients even when the discriminator is momentarily confused.

Results and Discussion

The paper evaluates MR-GAN using the Geometry Score (GS), a metric that compares the topological properties of real versus generated data (lower is better).

Synthetic Toy Data

On a task involving 8 Gaussians arranged in a circle but embedded in 3D space, standard GANs often suffer from mode collapse. MR-GAN captured all eight modes perfectly, improving the Geometry Score by 51.4%.

Figure 1: Visualization of the 8-Gaussian toy task. MR-GAN successfully captures the circular manifold structure, avoiding the mode collapse typical of standard GANs.

MNIST and CelebA Performance

On the MNIST dataset, adding the regularizer to a WGAN (MR-WGAN) reduced the GS from 0.414 to 0.372. The most significant improvements were seen on the complex CelebA dataset. MR-DCGAN achieved a GS of 6.33, compared to 10.77 for the standard DCGAN—a 40% improvement in capturing underlying data geometry. Qualitatively, the generated faces show higher diversity and fewer artifacts.

Figure 2: Synthetic face images generated by MR-DCGAN on the CelebA dataset, demonstrating high visual quality and diversity.

Beyond empirical results, the authors provide a theoretical proof that MR-GAN is locally exponentially stable. While popular variants like Wasserstein GAN (WGAN) can oscillate indefinitely, MR-GAN is proven to converge to a stable equilibrium point.

Conclusion

MR-GAN introduces a robust theoretical and practical framework for stabilizing GAN training by respecting the manifold structure of real data. While the paper establishes local stability, future work will investigate global convergence and the potential of joint regularization for both the generator and discriminator.

Summary updated

Abstract

Despite the growing interest in generative adversarial networks (GANs), training GANs remains a challenging problem, both from a theoretical and a practical standpoint. To address this challenge, in this paper, we propose a novel way to exploit the unique geometry of the real data, especially the manifold information. More specifically, we design a method to regularize GAN training by adding an additional regularization term referred to as manifold regularizer. The manifold regularizer forces the generator to respect the unique geometry of the real data manifold and generate high quality data. Furthermore, we theoretically prove that the addition of this regularization term in any class of GANs including DCGAN and Wasserstein GAN leads to improved performance in terms of generalization, existence of equilibrium, and stability. Preliminary experiments show that the proposed manifold regularization helps in avoiding mode collapse and leads to stable training.
BibTeX
@misc{li2018mr,
  author = {Qunwei Li and Bhavya Kailkhura and Rushil Anirudh and Yi Zhou and Yingbin Liang and Pramod K. Varshney},
  title = {{MR-GAN: Manifold Regularized Generative Adversarial Networks}},
  year = {2018},
  eprint = {1811.10427},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.10427},
}

Loading related content...

Figure