Abstract
Generative Adversarial Networks (GANs) are highly effective for image synthesis but suffer from training instabilities and mode collapse. Traditionally, researchers have addressed these issues by manually designing loss functions. This paper introduces TaylorGAN, an automated approach that discovers optimized loss functions through evolutionary metalearning. By parameterizing losses as third-degree Taylor polynomials and evolving them using the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES), TaylorGAN identifies formulations that surpass hand-designed standards like Wasserstein loss. On the CMP Facade dataset, TaylorGAN achieves significant improvements in both structural similarity (SSIM) and perceptual distance, producing images with enhanced textural realism and stability.
Introduction
Generative Adversarial Networks (GANs) have revolutionized the field of image synthesis, yet they remain notoriously difficult to train. The adversarial competition between a generator and a discriminator often leads to "brittle" dynamics, where small changes in hyperparameters or architecture can result in training failure or mode collapse. To stabilize this process, the research community has historically relied on human intuition to derive new loss functions, such as Minimax, Non-Saturating, and Wasserstein losses.
However, manual derivation is slow and may not capture the optimal mathematical dynamics for specific datasets. TaylorGAN addresses this by automating the design of loss functions, treating the search for an optimal loss as a metalearning problem.
Methodology: Taylor-Parameterized Genetic Loss Optimization
The core of TaylorGAN is the TaylorGLO (Taylor-parameterized Genetic Loss Optimization) technique. This method moves away from fixed logarithmic or linear formulas, instead representing loss functions as flexible third-degree multivariate Taylor polynomials.
The Evolutionary Search Process
The system evolves three distinct components of the GAN loss structure simultaneously:
- The discriminator’s loss for real samples.
- The discriminator’s loss for synthetic (fake) samples.
- The generator’s loss for synthetic samples.
To navigate this 12-parameter search space, the researchers employed the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES). This derivative-free genetic algorithm iteratively samples candidate loss functions, evaluates them through partial model training, and updates its internal mean vector and covariance matrix to converge on the most effective parameters.
Composite Objectives
To ensure the evolved losses produce high-quality images, the evolution is guided by a "Composite Objective"—a weighted sum of the Structural Similarity Index (SSIM) and Perceptual Distance (based on VGG-16 features). This multiobjective approach prevents the evolutionary process from over-optimizing a single metric at the expense of visual quality.
Experimental Setup
The approach was rigorously evaluated on the CMP Facade dataset, a benchmark for conditional image-to-image translation. The task involves generating photorealistic building images from semantic segmentation maps. The dataset consists of 606 perspective-corrected images, split into training (80%), validation (10%), and testing (10%) sets. This relatively small dataset size also served as a test for the robustness and regularization capabilities of the evolved loss functions.
Results and Discussion
TaylorGAN was compared against a state-of-the-art baseline: pix2pix-HD trained with Wasserstein loss and spectral normalization.
Quantitative Performance
The evolved loss functions demonstrated clear statistical superiority over the baseline after only 49 generations of evolution:
- Structural Similarity Index (SSIM): Improved from a baseline average of 9.4359 to 11.6615.
- Perceptual Distance: Reduced from 2129.5069 to 2040.2561, indicating higher visual fidelity.
Qualitative Analysis
Beyond the numbers, TaylorGAN produced visually distinct improvements. While baseline models often exhibited an unnatural "orange tint" or "plastic" textures, TaylorGAN images featured sharper details and more realistic material representations. There was also a more pronounced separation between foreground elements (like buildings) and the background (the sky), suggesting that the evolved losses better guide the discriminator in identifying fine-grained structural features.
Conclusion
TaylorGAN demonstrates that automated loss function discovery can significantly enhance GAN performance without requiring manual mathematical derivation. By treating loss design as an evolutionary optimization problem, the framework finds "mathematical sweet spots" that improve training stability and image quality. Future work will explore the generalizability of these evolved losses to unconditioned generation tasks and higher-resolution datasets.
Summary updated