Towards Good Practices for Data Augmentation in GAN Training

Ngoc-Trung Tran, Viet-Hung Tran, Ngoc-Bao Nguyen, Trung-Kien Nguyen, Ngai-Man CheungarXiv 2020 · arXiv

Abstract

Generative Adversarial Networks (GANs) typically require large datasets to achieve high-quality results. While classical data augmentation (DA) is a standard solution for data scarcity in supervised learning, it often degrades GAN performance by causing the generator to mimic the augmented distribution rather than the target data distribution. This paper introduces Data Augmentation Optimized for GAN (DAG), a principled framework that utilizes multiple discriminator branches and weight sharing. By ensuring transformations are handled correctly within the adversarial loop, DAG allows the model to benefit from augmented data without shifting the generative target. Our results across standard vision benchmarks and medical imaging datasets show significant improvements in Frechet Inception Distance (FID) and robustness to limited data.

Introduction

The success of GANs is heavily dependent on the availability of large-scale training data. In specialized fields like medical imaging, where data is scarce, researchers often apply classical DA techniques such as rotation, flipping, or cropping. However, in a standard GAN setup, the generator ($G$) aims to match the distribution of the training set. If the training set is augmented, $G$ learns to produce augmented samples (e.g., rotated digits or flipped faces), which is often undesirable and leads to higher FID scores.

The core challenge is to leverage augmented data to improve the discriminator's ($D$) feature representation without misleading the generator. The authors propose that by using multiple discriminator branches and carefully managing transformations, one can provide a richer training signal to $G$ while keeping it focused on the original data distribution $P_d$.

Methods: The DAG Framework

The Data Augmentation Optimized for GAN (DAG) framework is designed around three key components:

  1. Multiple Discriminator Branches: Instead of a single discriminator, DAG employs a set of discriminators $\{D, D_2, \dots, D_K\}$. Each branch $D_k$ is responsible for a specific transformation $T_k$.
  2. Invertible Transformations: The framework relies on the theoretical insight that if a transformation $T$ is differentiable and invertible, the Jensen-Shannon divergence is preserved. By transforming the generator's output $G(z)$ before passing it to the corresponding discriminator branch $D_k$, the feedback remains consistent with the original distribution.
  3. Weight-Sharing Backbone: To manage computational complexity and encourage robust feature learning, all discriminators share a common backbone. Distinct "heads" are used for each transformation task, forcing the shared layers to learn invariant features across all augmented versions of the data.
The DAG architecture utilizes multiple discriminator branches and a weight-sharing backbone to process various image transformations.

Experimental Results

The authors evaluated DAG on several benchmarks, including MNIST, CIFAR-10/100, STL-10, and the IXI MRI dataset.

Performance on Vision Benchmarks

DAG consistently outperformed baseline models and standard DA approaches. When integrated with MS-DistGAN, it achieved state-of-the-art FID scores on CIFAR-10 (13.20) and STL-10 (25.56). The framework proved particularly effective in low-data regimes; for instance, on a 10% subset of CIFAR-10, DAG reduced the FID of BigGAN from 48.3 to 36.9.

Robustness Across Augmentation Types

A key strength of DAG is its stability across different types of transformations. While classical DA can be actively harmful (e.g., causing MNIST FID to jump from 6.8 to 73.1 due to unwanted rotations), DAG maintains or improves performance regardless of the augmentation used.

FID scores over training iterations for various augmentation types, comparing standard SS-GAN and Dist-GAN against their DAG-enhanced versions.

Medical Imaging and Image-to-Image Translation

On the IXI MRI dataset with only 1,000 samples, DAG reduced the FID from 71.12 to 46.83. It also showed versatility in image-to-image tasks, improving the Class IOU on the Cityscapes dataset within a CycleGAN architecture from 0.02 to 0.15.

Conclusion

DAG provides a robust methodology for incorporating data augmentation into GAN training. By using multiple discriminator heads and preserving distributional properties through transformation-aware training, it overcomes the "distribution shift" problem inherent in classical DA. While the approach increases training time (2x to 5x), the significant gains in image quality and data efficiency make it a valuable tool for training GANs on limited datasets. Future work may explore extending the theoretical guarantees to non-invertible transformations and optimizing the hyperparameter tuning process.

Summary updated

Abstract

Recent successes in Generative Adversarial Networks (GAN) have affirmed the importance of using more data in GAN training. Yet it is expensive to collect data in many domains such as medical applications. Data Augmentation (DA) has been applied in these applications. In this work, we first argue that the classical DA approach could mislead the generator to learn the distribution of the augmented data, which could be different from that of the original data. We then propose a principled framework, termed Data Augmentation Optimized for GAN (DAG), to enable the use of augmented data in GAN training to improve the learning of the original distribution. We provide theoretical analysis to show that using our proposed DAG aligns with the original GAN in minimizing the Jensen-Shannon (JS) divergence between the original distribution and model distribution. Importantly, the proposed DAG effectively leverages the augmented data to improve the learning of discriminator and generator. We conduct experiments to apply DAG to different GAN models: unconditional GAN, conditional GAN, self-supervised GAN and CycleGAN using datasets of natural images and medical images. The results show that DAG achieves consistent and considerable improvements across these models. Furthermore, when DAG is used in some GAN models, the system establishes state-of-the-art Frechet Inception Distance (FID) scores. Our code is available.
BibTeX
@misc{tran2020towards,
  author = {Ngoc-Trung Tran and Viet-Hung Tran and Ngoc-Bao Nguyen and Trung-Kien Nguyen and Ngai-Man Cheung},
  title = {{Towards Good Practices for Data Augmentation in GAN Training}},
  year = {2020},
  eprint = {2006.05338},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.05338},
}
Figure