Generative Adversarial Networks are special cases of Artificial Curiosity (1990) and also closely related to Predictability Minimization (1991)

Jürgen Schmidhuber

TL;DR

This paper argues that Generative Adversarial Networks (GANs) are not entirely novel but are, in fact, a specific instance of a broader concept called Artificial Curiosity (AC) developed in 1990. It also highlights a close relationship between GANs and Predictability Minimization (PM), another technique from the early 1990s, aiming to clarify historical contributions and connections in unsupervised learning.

What Problem Does This Paper Address?

The paper addresses the historical context and conceptual lineage of Generative Adversarial Networks (GANs). While GANs have achieved remarkable success in generating realistic data, the authors contend that their underlying principles are not entirely new. They aim to correct the perception that GANs are a completely novel invention by tracing their roots back to earlier work on unsupervised learning and adversarial systems, specifically Artificial Curiosity (AC) and Predictability Minimization (PM). This clarification is important for understanding the evolution of machine learning and giving credit to foundational ideas.

Methods

The paper reviews and connects three key unsupervised learning paradigms:

  • Artificial Curiosity (AC, 1990): This approach involves two neural networks. One network (the controller) generates outputs, and the other (the world model) learns to predict the consequences of these outputs. The controller is trained to maximize the prediction errors of the world model, effectively seeking novel situations that the world model finds surprising. This creates an intrinsic reward signal for exploration.
  • Predictability Minimization (PM, 1990s): PM focuses on learning disentangled, factorial codes of data. It uses an encoder network that maps data to a code, and a predictor network that tries to predict components of the code from the others. The encoder is trained to maximize the predictor's errors, forcing the code components to become statistically independent and informative.
  • Generative Adversarial Networks (GANs, 2010-2014): The paper posits that GANs are a special case of AC. In this setup, the "environment" is simplified: it checks if a generated output belongs to a predefined set of "real" data. The generator (controller) tries to produce outputs that fool the discriminator (world model), while the discriminator tries to distinguish real from fake data.

The core methodology involves explaining how these systems operate as minimax games, where one network's objective is to maximize the loss function minimized by the other. The paper emphasizes the use of gradient descent and backpropagation for training these adversarial networks.

Datasets

Not discussed in the paper. The paper focuses on theoretical connections and algorithmic principles rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • GANs as a Special Case of AC: The most significant insight is that GANs can be viewed as a simplified instantiation of the Artificial Curiosity framework from 1990. The key difference lies in the nature of the "environment" and the "prediction task."
  • Historical Roots of Adversarial Learning: The paper highlights that adversarial training principles for unsupervised learning were established much earlier than commonly perceived, predating GANs by decades.
  • Predictability Minimization's Role: PM is presented as a related but distinct approach that also uses adversarial minimax principles to learn data representations, specifically aiming for disentangled factorial codes.
  • Duality between PM and GANs: A conceptual duality is drawn between PM and GANs, where PM's encoder learns to map data to codes, and GANs' generator learns to map codes to data. This suggests a potential for cross-pollination of ideas.
  • Clarifying Misconceptions: The paper explicitly corrects a previously published claim that PM is not based on a minimax game, asserting that it is indeed a pure minimax game.

Key Results

The paper does not present quantitative results from experiments. Its contribution is primarily conceptual and historical, clarifying the relationships between different unsupervised learning paradigms. The "results" are the insights into how GANs, AC, and PM are interconnected through their shared adversarial minimax structure.

Limitations and Future Work

  • Generalization of AC: The paper notes that while GANs are a specific application, the broader AC framework has much wider applicability, especially in sequential decision-making and reinforcement learning in partially observable environments.
  • Experimental Validation: The paper is largely theoretical and historical. While it draws connections, it doesn't present new experimental results to empirically validate these connections or demonstrate novel capabilities.
  • Convergence Proofs: The paper briefly mentions that convergence proofs for GANs can be transferred to PM, but it doesn't delve into the specifics of these proofs or their implications for practical training.
  • Future Research: The paper suggests it would be interesting to experimentally study whether the GAN pipeline (code to data) is easier to train than PM (data to code) for learning factorial codes or useful approximations thereof. It also hints that the large body of recent GAN-related insights might help improve training procedures for AC systems.

Summary updated

BibTeX
@article{schmidhuber2020generative,
  author = {Jürgen Schmidhuber},
  title = {{Generative Adversarial Networks are special cases of Artificial Curiosity (1990) and also closely related to Predictability Minimization (1991)}},
  journal = {Neural Networks},
  year = {2020},
  doi = {10.1016/j.neunet.2020.04.008},
}
Figure