Abstract
Generative Adversarial Networks (GANs) are a powerful class of deep generative models. In this paper, we extend GAN to the problem of generating data that are not only close to a primary data source but also required to be different from auxiliary data sources. For this problem, we enrich both GANs' formulations and applications by introducing pushing forces that thrust generated samples away from given auxiliary data sources. We term our method Push-and-Pull GAN (P2GAN). We conduct extensive experiments to demonstrate the merit of P2GAN in two applications: generating data with constraints and addressing the mode collapsing problem. We use CIFAR-10, STL-10, and ImageNet datasets and compute Fréchet Inception Distance to evaluate P2GAN's effectiveness in addressing the mode collapsing problem. The results show that P2GAN outperforms the state-of-the-art baselines. For the problem of generating data with constraints, we show that P2GAN can successfully avoid generating specific features such as black hair.
Introduction
Standard Generative Adversarial Networks (GANs) typically learn from a single data source, where a generator attempts to match a target distribution by fooling a discriminator. However, this paradigm faces two significant challenges: mode collapse and single-source limitations. Mode collapse occurs when the generator produces a limited variety of samples, failing to capture the full diversity of the target distribution. Furthermore, standard GANs lack a mechanism to explicitly avoid certain features without complex, labeled conditional setups.
To address these issues, we introduce Push-and-Pull GAN (P2GAN). This framework allows models to learn from multiple data sources simultaneously by balancing "pulling" forces (attraction to the target) and "pushing" forces (repulsion from unwanted data).
Methods: The Push-and-Pull Formulation
The core innovation of P2GAN is the extension of the standard $f$-divergence objective to include auxiliary sources.
- The Pull: A primary discriminator $S(x)$ provides a "pulling force," encouraging the generator $G$ to match the primary data distribution.
- The Push: One or more auxiliary discriminators $\tilde{S}_i(x)$ provide "pushing forces," rewarding the generator for producing samples that are distinct from auxiliary distributions.
Addressing Mode Collapse
To solve mode collapse, we employ a sequential training strategy. After training an initial generator, its output is treated as an auxiliary source. Subsequent generators are then "pushed" away from these previously learned distributions, forcing them to explore and capture missing modes of the data.
Constrained Generation
P2GAN enables unsupervised constrained generation. For example, to generate images with a specific feature (e.g., blond hair) from a mixed dataset, an auxiliary set containing the unwanted feature (e.g., black hair) can be used to "push" the generator toward the desired attribute. Unlike conditional GANs, this does not require labeling the primary dataset.
Key Results and Discussion
P2GAN was evaluated on synthetic 2D data, CIFAR-10, STL-10, ImageNet, and CelebA.
Quantitative Performance and Stability
P2GAN achieved state-of-the-art Fréchet Inception Distance (FID) scores. On CIFAR-10, it reached an FID of 20.1, outperforming SN-GAN (21.7) and MGAN (20.9). On STL-10, it achieved 38.2, compared to SN-GAN's 40.1.
Crucially, P2GAN exhibits superior training stability. While standard GANs often become unstable after many epochs, P2GAN's performance remains steady and continues to improve over time.
Qualitative Success
In the CelebA experiments, P2GAN successfully generated blond-haired faces by being pushed away from a black-haired auxiliary set. This demonstrates the effectiveness of the pushing force in steering the generator toward specific semantic regions of the data space without explicit labels.
Conclusion
P2GAN enriches the GAN framework by introducing pushing forces, providing a robust solution to mode collapse and a flexible method for constrained generation. While the pushing force hyperparameter requires tuning and sequential training adds computational cost, the gains in stability and diversity are significant. Future work will focus on more efficient single-generator architectures and applications in anomaly detection.
Summary updated