Dropout Induced Noise for Co-Creative GAN Systems

Sabine Wieluch, Friedhelm SchwenkerWorkshops @ ICCV 2019 · DOI · arXiv

Abstract

This paper explores the use of Dropout during the inference phase of Generative Adversarial Networks (GANs) to generate diverse outputs from a single input. In conditional GAN tasks, generators often ignore traditional noise vectors, leading to deterministic outputs. By maintaining Dropout at test time, particularly in early layers, the authors demonstrate a method to induce "creative" noise that respects structural constraints while providing meaningful variety. Experiments on MNIST and CMP Facades datasets show that this approach offers a robust alternative to latent space exploration for co-creative AI applications.

Introduction

In co-creative systems where humans and AI collaborate, users often require multiple suggestions from a single starting point. For instance, an architect providing a building silhouette might expect the AI to suggest various window and door configurations. Standard GANs typically utilize a noise vector to create variety; however, in conditional GANs (cGANs) used for tasks like image inpainting or style transfer, the generator frequently learns to ignore this noise, producing nearly identical results. While latent space exploration is a common workaround, it can be difficult to control and may inadvertently alter the core constraints of the input. This research seeks a method to induce "creative" noise that respects the input's structure while providing meaningful variety.

Methodology

The authors propose a simple yet effective modification: keeping Dropout active during the generation (inference) phase. While Dropout is traditionally a regularization tool used only during training to prevent neuron co-adaptation, the authors experiment with several configurations of "Inference-time Dropout":

  • Inverted Dropout: Utilizing standard implementations where scaling occurs during training so the generation phase requires no mathematical alterations.
  • Layer Targeting: Applying Dropout to all hidden layers versus applying it only to the first hidden layer to allow subsequent layers to "repair" the noise.
  • Scaling Factors: Testing the necessity of the typical $1/(1-p)$ scaling factor to maintain the expected activation distribution.

To quantify variety, the researchers generated 500 images from the same input and calculated the standard deviation of the Euclidean distances between the standard (no-dropout) output and the stochastic (dropout-enabled) outputs.

Results and Discussion

Quantitative and Qualitative Variety

In MNIST experiments, the variety metric increased significantly with the generation dropout rate ($p$). For a model trained with $p=0.8$, the variety jumped from 0 (at $p=0$ in generation) to 5.213 (at $p=0.8$ in generation). Qualitatively, the model successfully generated multiple valid variations of the same input digit.

Figure 4: Variations of the digit '9' generated using inference-time dropout, demonstrating the model's ability to produce diverse yet recognizable outputs.

The "Repair" Effect and Image Breakage

A critical finding is that applying Dropout only to the early layers allows subsequent layers to "repair" the induced noise, resulting in structurally coherent images. Conversely, models trained without Dropout cannot handle Dropout during generation; removing even a few units causes the process to collapse into visual gibberish. Furthermore, skipping the scaling factor leads to under-saturation and extreme noise.

Figure 5: Examples of image "breakage" where high dropout rates or lack of scaling leads to a loss of structural coherence and the introduction of extreme noise.

On the CMP Facades dataset, the model successfully varied the size and placement of architectural elements like shops and cornices while keeping original window constraints intact. However, a trade-off exists: higher dropout rates increase variety and edge sharpness but risk losing fine details or producing incoherent images.

Conclusion

Dropout-induced noise provides a simple, effective mechanism for generating diverse outputs in conditional GANs. While it offers a robust alternative to latent space exploration, the primary challenge remains the trade-off between variety and coherence. Future work will focus on finding the "sweet spot" for dropout rates across different architectures and optimizing non-dropout layers to better repair creative noise introduced in earlier stages.

Summary updated

BibTeX
@inproceedings{wieluch2019dropout,
  author = {Sabine Wieluch and Friedhelm Schwenker},
  title = {{Dropout Induced Noise for Co-Creative GAN Systems}},
  booktitle = {Workshops @ ICCV},
  year = {2019},
  doi = {10.1109/iccvw.2019.00383},
}
Figure