Abstract
Generative Adversarial Networks (GANs) typically require large datasets to achieve high-quality results. While classical data augmentation (DA) is a standard solution for data scarcity in supervised learning, it often degrades GAN performance by causing the generator to mimic the augmented distribution rather than the target data distribution. This paper introduces Data Augmentation Optimized for GAN (DAG), a principled framework that utilizes multiple discriminator branches and weight sharing. By ensuring transformations are handled correctly within the adversarial loop, DAG allows the model to benefit from augmented data without shifting the generative target. Our results across standard vision benchmarks and medical imaging datasets show significant improvements in Frechet Inception Distance (FID) and robustness to limited data.
Introduction
The success of GANs is heavily dependent on the availability of large-scale training data. In specialized fields like medical imaging, where data is scarce, researchers often apply classical DA techniques such as rotation, flipping, or cropping. However, in a standard GAN setup, the generator ($G$) aims to match the distribution of the training set. If the training set is augmented, $G$ learns to produce augmented samples (e.g., rotated digits or flipped faces), which is often undesirable and leads to higher FID scores.
The core challenge is to leverage augmented data to improve the discriminator's ($D$) feature representation without misleading the generator. The authors propose that by using multiple discriminator branches and carefully managing transformations, one can provide a richer training signal to $G$ while keeping it focused on the original data distribution $P_d$.
Methods: The DAG Framework
The Data Augmentation Optimized for GAN (DAG) framework is designed around three key components:
- Multiple Discriminator Branches: Instead of a single discriminator, DAG employs a set of discriminators $\{D, D_2, \dots, D_K\}$. Each branch $D_k$ is responsible for a specific transformation $T_k$.
- Invertible Transformations: The framework relies on the theoretical insight that if a transformation $T$ is differentiable and invertible, the Jensen-Shannon divergence is preserved. By transforming the generator's output $G(z)$ before passing it to the corresponding discriminator branch $D_k$, the feedback remains consistent with the original distribution.
- Weight-Sharing Backbone: To manage computational complexity and encourage robust feature learning, all discriminators share a common backbone. Distinct "heads" are used for each transformation task, forcing the shared layers to learn invariant features across all augmented versions of the data.
Experimental Results
The authors evaluated DAG on several benchmarks, including MNIST, CIFAR-10/100, STL-10, and the IXI MRI dataset.
Performance on Vision Benchmarks
DAG consistently outperformed baseline models and standard DA approaches. When integrated with MS-DistGAN, it achieved state-of-the-art FID scores on CIFAR-10 (13.20) and STL-10 (25.56). The framework proved particularly effective in low-data regimes; for instance, on a 10% subset of CIFAR-10, DAG reduced the FID of BigGAN from 48.3 to 36.9.
Robustness Across Augmentation Types
A key strength of DAG is its stability across different types of transformations. While classical DA can be actively harmful (e.g., causing MNIST FID to jump from 6.8 to 73.1 due to unwanted rotations), DAG maintains or improves performance regardless of the augmentation used.
Medical Imaging and Image-to-Image Translation
On the IXI MRI dataset with only 1,000 samples, DAG reduced the FID from 71.12 to 46.83. It also showed versatility in image-to-image tasks, improving the Class IOU on the Cityscapes dataset within a CycleGAN architecture from 0.02 to 0.15.
Conclusion
DAG provides a robust methodology for incorporating data augmentation into GAN training. By using multiple discriminator heads and preserving distributional properties through transformation-aware training, it overcomes the "distribution shift" problem inherent in classical DA. While the approach increases training time (2x to 5x), the significant gains in image quality and data efficiency make it a valuable tool for training GANs on limited datasets. Future work may explore extending the theoretical guarantees to non-invertible transformations and optimizing the hyperparameter tuning process.
Summary updated