The Bures Metric for Generative Adversarial Networks

Hannes De Meulemeester, Joachim Schreurs, Michaël Fanuel, Bart De Moor, Johan A. K. SuykensECML/PKDD 2021 · DOI

Abstract

Mode collapse remains a significant challenge in training Generative Adversarial Networks (GANs). We present BuresGAN, a novel regularization technique that utilizes the Bures metric to enforce diversity in generated samples. By matching the covariance matrices of real and generated data in the discriminator's feature space, BuresGAN effectively mitigates mode collapse. Our results on synthetic and real-world datasets demonstrate superior mode coverage and competitive image quality with minimal computational overhead.

Introduction

Generative Adversarial Networks (GANs) have revolutionized the field of generative modeling, yet they remain notoriously difficult to train. A primary failure mode is mode collapse, where the generator learns to produce only a limited subset of the target distribution's variety. While various diversity-promoting penalties have been proposed, many are computationally expensive or require complex architectural modifications. This paper introduces a mathematically grounded and efficient solution: the Bures metric.

Methodology: Diversity Matching via the Bures Metric

The core innovation of BuresGAN is the use of the Bures distance to quantify and minimize the difference in diversity between real and generated data.

  1. Feature Space Representation: The method utilizes activations from the discriminator's final layer as a high-dimensional feature space.
  2. Covariance Matching: The authors propose that matching the covariance structure of features in this space is a powerful proxy for matching data diversity.
  3. The Bures Penalty: A squared Bures distance penalty is added to the generator's loss function. This Riemannian metric measures the distance between positive semi-definite (covariance) matrices.
  4. Computational Efficiency: To avoid expensive eigendecompositions, the implementation uses the Newton-Schultz algorithm for matrix square roots. Furthermore, a duality between feature and kernel spaces allows the method to scale efficiently regardless of whether the batch size or feature dimension is larger.

Experimental Results

The effectiveness of BuresGAN was evaluated on several benchmarks specifically designed to test mode recovery.

Synthetic Benchmarks

On 2D synthetic datasets like "RING" (8 modes) and "GRID" (25 modes), BuresGAN demonstrated a consistent ability to capture all target modes. As shown in the training progression below, the model successfully spreads its generated distribution from a central cluster to cover every individual mode without collapse.

Training progression of BuresGAN on RING (top) and GRID (bottom) synthetic datasets.

Real-World Datasets

  • Stacked MNIST: In this task featuring 1,000 distinct modes, BuresGAN recovered an average of 997.1 modes, significantly outperforming previous methods like VEEGAN and Unrolled GAN.
  • Natural Images: On CIFAR-10, BuresGAN achieved an Inception Score (IS) of 8.81 and a Fréchet Inception Distance (FID) of 12.91. These results are particularly notable because the Bures distance is mathematically related to the FID score, meaning the model directly optimizes for the metric used to evaluate it.

Discussion and Conclusion

BuresGAN provides a robust, efficient, and theoretically sound approach to stabilizing GAN training. By focusing on covariance matching in the feature space, it ensures that the generator maintains the diversity inherent in the training data. The method's stability across different architectures and its minimal impact on training speed make it a practical tool for improving generative models. Future work may involve integrating this diversity term into larger-scale architectures like StyleGAN to further enhance their stability and sample variety.

Summary updated

BibTeX
@inproceedings{meulemeester2021bures,
  author = {Hannes De Meulemeester and Joachim Schreurs and Michaël Fanuel and Bart De Moor and Johan A. K. Suykens},
  title = {{The Bures Metric for Generative Adversarial Networks}},
  booktitle = {ECML/PKDD},
  year = {2021},
  doi = {10.1007/978-3-030-86520-7\_4},
}
Figure