Abstract
In adversarial learning, the discriminator often fails to guide the generator successfully because it distinguishes between real and generated images using non-robust or "silly" features. This paper presents a simple yet effective way to improve Generative Adversarial Network (GAN) performance without imposing training overhead or modifying existing network architectures. The proposed method employs a novel Cascading Rejection (CR) module for the discriminator, which iteratively extracts multiple non-overlapping features using a vector rejection operation. By preventing the discriminator from concentrating on a single set of features, the CR module guides the generator to produce images more similar to the real data distribution. Quantitative evaluations across datasets like CIFAR-10, CelebA, and LSUN show significant improvements in Frechet Inception Distance (FID), confirming enhanced image diversity and quality.
The Problem: Discriminator "Laziness"
Generative Adversarial Networks (GANs) are notoriously difficult to train. A common failure mode is that the discriminator becomes "lazy." In a standard GAN, the final layer typically performs an inner product between a weight vector and an image feature vector to produce a single score.
Mathematically, this means the discriminator only focuses on the part of the feature space parallel to that weight vector, completely ignoring the "perpendicular" feature space. If the discriminator identifies a non-robust feature (like a specific pixel artifact) that distinguishes real from fake, it stops learning. Consequently, the generator is never pressured to learn the complex, high-dimensional details of the actual data distribution, leading to poor image quality or "mode collapse."
The Cascading Rejection (CR) Module
To force the discriminator to be more thorough, the authors propose the Cascading Rejection (CR) module. This module acts as a lightweight "plug-and-play" addition that forces the discriminator to look at multiple, non-overlapping feature sets.
- Vector Rejection: Instead of producing one scalar value, the CR module works iteratively. After the first inner product, it uses a vector rejection operation to subtract the part of the feature vector that the discriminator just used.
- Iterative Extraction: This "leftover" feature vector is then passed to another inner product with a different weight vector. This process repeats $N$ times.
- Non-Overlapping Features: Because of the rejection step, each subsequent scalar value is derived from a part of the feature space that is orthogonal (perpendicular) to the previous ones.
- Modified Loss: The final adversarial loss is calculated by averaging the results of these $N$ different "views" of the image.
This method requires only a few basic vector operations at the end of the discriminator, adding virtually zero computational cost and requiring no changes to the rest of the network's architecture.
Key Insights and Results
By forcing the discriminator to look at $N$ different directions in the feature space, the generator can no longer "cheat" by fixing one specific artifact. It must improve the image across all $N$ feature dimensions to fool the discriminator. This orthogonality ensures that each "head" of the CR module is looking at a unique, non-overlapping part of the data.
Quantitative Performance
The primary metric used was the Frechet Inception Distance (FID), where a lower score indicates better quality and diversity. Key results include:
- CIFAR-10: FID improved from 16.02 (traditional GAN) to 13.90 ($N=16$).
- LSUN: FID dropped from 19.39 to 16.77.
- High-Resolution Success: On CelebA-HQ (512x512), the CR module improved the FID from 32.63 to 30.82.
- Zero Overhead: Training time per iteration remained almost identical to the baseline, making it a "free" upgrade for most GAN pipelines.
Conclusion
The Cascading Rejection module provides a simple, architecture-agnostic way to stabilize GAN training and improve image synthesis. While the method currently focuses on the final layer of the discriminator, its success suggests that the principle of cascading rejection could be applied to other areas of deep learning where diverse feature extraction is beneficial, such as self-supervised learning.
Summary updated