Discriminative Forests Improve Generative Diversity for Generative Adversarial Networks

Junjie Chen, Jiahao Li, Chen Song, Bin Li, Qingcai Chen, Hongchang Gao, Wendy Hui Wang, Zenglin Xu, Xinghua Shi

Introduction

Generative Adversarial Networks (GANs) rely on a competitive dynamic between a Generator and a Discriminator. To produce high-quality, diverse images, the discriminator must possess high capacity. However, powerful discriminators often overfit by memorizing training samples rather than learning the underlying distribution. This leads to mode collapse, where the generator produces limited variations to exploit the discriminator's weaknesses. Previous attempts to use multiple discriminators often struggled because they split data into smaller, less representative chunks.

The Forest-GAN Framework

To address this, researchers introduced Forest-GAN, a framework that replaces the single discriminator with a "Discriminative Forest" of $K$ independent discriminators. Inspired by the classic Random Forest algorithm, this approach leverages the "wisdom of the crowd" to improve generalization.

The Forest-GAN architecture: A generator competes against an ensemble of discriminators trained on bootstrapped data subsets.

The process involves three main steps:

  1. Bootstrapping: Each discriminator $D_k$ is trained on a bootstrapped version of the real dataset $X$ (sampling with replacement). This ensures each discriminator sees a slightly different but full-sized subset of the data.
  2. Independent Training: The discriminators are initialized and updated independently.
  3. Aggregation: Feedback to the generator is provided by averaging the outputs or gradients of all $K$ discriminators.

Theoretically, the forest's generalization error is bounded by the individual discriminators' strength and their mutual correlation. By maintaining diversity (low correlation) through bootstrapping, the forest can scale in capacity without the risk of overfitting.

Key Insights and Mode Recovery

The core advantage of Forest-GAN is its natural defense against overfitting. Since no single discriminator sees the entire "perfect" dataset, the generator cannot easily find a single loophole to cheat. Increasing the number of discriminators ($K$) actually improves robustness and diversity, contrary to standard GANs where more parameters often lead to faster overfitting.

Performance on 2D-GMM: As $K$ increases, the model recovers more modes of the target distribution, significantly improving precision and recall over time.

On a simulated 2D Gaussian Mixture Model (2D-GMM) with 9 modes, a standard GAN ($K=1$) captured only 4 modes. As shown in the figure above, increasing $K$ to 50 allowed Forest-GAN to recover all 9 modes with near-perfect precision and recall. This demonstrates the model's ability to approximate a mixture of bootstrapped distributions, forcing the generator to learn a more generalized representation.

Experimental Results

Forest-GAN was evaluated on real-world datasets using AutoGAN and StyleGAN2-ADA baselines, delivering substantial improvements:

  • STL10: Improved the Frechet Inception Distance (FID)—a measure of image quality and diversity—from 30.71 down to 19.27.
  • LSUN-cat: Reduced FID from 9.22 to 6.87, outperforming models trained on much larger datasets.
  • Diversity Metrics: The model achieved significantly higher Recall and lower Perceptual Path Length (PPL), indicating smoother style transitions and greater variety in generated samples.

Limitations and Future Work

While effective, training $K$ discriminators increases computational and memory overhead. Additionally, because all discriminators face the same generator, they may become more correlated over time, which can slightly degrade the theoretical error bounds. Future research may explore "feature bootstrapping" (partitioning image data) or more advanced aggregation methods to further decouple the ensemble and boost performance.

Summary updated

Abstract

Improving the diversity of Artificial Intelligence Generated Content (AIGC) is one of the fundamental problems in the theory of generative models such as generative adversarial networks (GANs). Previous studies have demonstrated that the discriminator in GANs should have high capacity and robustness to achieve the diversity of generated data. However, a discriminator with high capacity tends to overfit and guide the generator toward collapsed equilibrium. In this study, we propose a novel discriminative forest GAN, named Forest-GAN, that replaces the discriminator to improve the capacity and robustness for modeling statistics in real-world data distribution. A discriminative forest is composed of multiple independent discriminators built on bootstrapped data. We prove that a discriminative forest has a generalization error bound, which is determined by the strength of individual discriminators and the correlations among them. Hence, a discriminative forest can provide very large capacity without any risk of overfitting, which subsequently improves the generative diversity. With the discriminative forest framework, we significantly improved the performance of AutoGAN with a new record FID of 19.27 from 30.71 on STL10 and improved the performance of StyleGAN2-ADA with a new record FID of 6.87 from 9.22 on LSUN-cat.
BibTeX
@inproceedings{chen2024discriminative,
  author = {Junjie Chen and Jiahao Li and Chen Song and Bin Li and Qingcai Chen and Hongchang Gao and Wendy Hui Wang and Zenglin Xu and Xinghua Shi},
  title = {{Discriminative Forests Improve Generative Diversity for Generative Adversarial Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i10.29013},
}
Figure