Abstract
Generative Adversarial Networks (GANs) have swiftly evolved to imitate increasingly complex image distributions. However, the majority of developments focus on the performance of GANs on balanced datasets. In real-world scenarios, data often follows a long-tailed distribution, where a few majority classes dominate. Standard GANs trained on such data tend to ignore minority classes, leading to mode collapse. This paper proposes a novel class-balancing regularizer that incorporates a pre-trained classifier to monitor and adjust the generator's output distribution. By tracking an "effective class frequency" via an exponential moving average, the method penalizes over-generation of majority classes and encourages diversity. Experiments on datasets like iNaturalist-2019 demonstrate significant improvements in both image quality (FID) and class balance.
Introduction
While GANs have achieved unprecedented success in image synthesis, most architectures are designed for and evaluated on perfectly balanced datasets. In practice, data is usually "long-tailed," meaning a few classes (the head) have abundant examples while many classes (the tail) have very few. When standard unconditional GANs are trained on long-tailed data, they often suffer from "missing modes," generating images almost exclusively from the majority classes. Conditional GANs (cGANs) can force the generation of specific classes, but they often suffer from degraded image quality and training instability when class imbalance is severe. This research addresses the challenge of forcing a GAN to learn a uniform, balanced distribution of classes even when its training data is highly imbalanced.
Methodology
To solve the imbalance problem, the authors propose a Class Balancing Regularizer that puts a "classifier in the loop."
As illustrated in the framework above, the system utilizes a pre-trained classifier to predict labels for generated batches. The framework tracks the effective class frequency—an exponential moving average of how many images of each class the GAN produces over time. This tracking is crucial because GANs dynamically shift their outputs and can "forget" minority classes during training.
A novel loss term is added to the generator's objective, calculating the weighted entropy of the batch's predicted class distribution. This loss is inversely weighted by the historical effective class frequency. If the GAN has recently generated too many majority-class samples, the regularizer heavily penalizes further majority-class generations, forcing the generator to synthesize underrepresented minority classes to minimize loss.
Experimental Results
The method was evaluated on several datasets, including CIFAR-10/100 and the naturally long-tailed iNaturalist-2019 dataset.
As shown in the comparison above, while standard models like SNDCGAN and ACGAN largely mirror the imbalanced training distribution, the proposed method ("Ours") successfully generates a near-uniform distribution across all classes. Key results include:
- Quality Improvements: On the iNaturalist-2019 dataset, the proposed method improved the Fréchet Inception Distance (FID) from 13.03 (baseline) to 9.01.
- True Class Balance: The method achieved significantly lower KL Divergence against a uniform distribution compared to baselines, proving that generated samples are evenly distributed.
- Semi-Supervised Robustness: The framework remains effective even when the pre-trained classifier is fine-tuned on as little as 0.1% of labeled data, outperforming fully unsupervised baselines.
Conclusion
The "classifier in the loop" approach effectively mitigates mode collapse in long-tailed scenarios by dynamically penalizing over-represented classes. While the method relies on the auxiliary classifier having at least a weak signal for tail classes (around 20% accuracy), it provides a stable and effective mechanism for achieving class-balanced generation without sacrificing image quality.
Summary updated