Unsupervised Minimax: Adversarial Curiosity, Generative Adversarial Networks, and Predictability Minimization

Jürgen Schmidhuber

Abstract

While Generative Adversarial Networks (GANs) are often viewed as a revolutionary breakthrough from 2014, their theoretical foundations lie in the unsupervised minimax frameworks developed in the early 1990s. This paper provides a historical and conceptual synthesis of adversarial learning, tracing the lineage of GANs back to Artificial Curiosity (AC) and Predictability Minimization (PM). By framing GANs as a specific, "blind" application of these earlier principles, we establish a unified perspective on how zero-sum games between neural networks drive the discovery of complex data distributions and independent features.

Introduction

In the mid-2010s, the emergence of Generative Adversarial Networks (GANs) transformed the landscape of machine learning, offering a powerful new way to model high-dimensional data. However, the core mechanism—two neural networks engaged in a minimax game—is not a recent invention. This paper argues that GANs are a subset of a broader principle known as Unsupervised Minimax, a framework pioneered by Jürgen Schmidhuber in 1990. By revisiting the origins of Artificial Curiosity and Predictability Minimization, we can better understand the mathematical engine that powers modern adversarial learning and correct historical misconceptions regarding its development.

The Three Pillars of Unsupervised Minimax

1. Artificial Curiosity (AC, 1990)

The earliest iteration of this framework, Artificial Curiosity, was designed for agents operating in unknown environments. The system consists of two components: a World Model (M) and a Controller (C).

  • The World Model (M) attempts to minimize its prediction error, striving to accurately represent the environment's transitions.
  • The Controller (C) is rewarded for actions that maximize M’s error.

This adversarial relationship forces the agent to seek out "interesting" or unpredictable parts of the world. Once a phenomenon is understood and becomes predictable, the error drops, and the controller—driven by the need for novelty—moves on to new challenges. This effectively creates a self-supervised exploration loop.

2. Predictability Minimization (PM, 1991)

Predictability Minimization applied the minimax principle to the problem of representation learning. In PM, an Encoder creates a code for incoming data, while a Predictor attempts to guess specific components of that code based on the others.

To create an efficient, non-redundant representation (a "factorial code"), the Encoder is trained to maximize the Predictor's error. This competition forces the Encoder to eliminate statistical dependencies between code components, resulting in independent feature detectors. Early experiments with PM successfully discovered visual features similar to those found in the human visual cortex, such as orientation-sensitive cells, without any labeled supervision.

3. Generative Adversarial Networks (GANs, 2014)

Schmidhuber frames GANs as a specific application of Artificial Curiosity where the "environment" is a static dataset. In this context, the Generator acts as a "blind" Controller that receives only noise as input. Its goal is to maximize the error of the Discriminator (the World Model), which is tasked with distinguishing real data from the Generator's synthetic outputs. The standard GAN training procedure—backpropagating the discriminator's error through to the generator—was a technique already established in the 1990 AC literature.

Key Theoretical Insights

The Boredom Principle and Learning Progress

A critical challenge in minimax systems is the "white noise problem." In a purely stochastic environment, a model will always have high prediction error, which could lead a naive controller to get "stuck" staring at random noise. To solve this, Schmidhuber introduced the concept of rewarding the first derivative of error—or learning progress. If the model is no longer improving its predictions, the controller receives no reward, effectively becoming "bored" and seeking new patterns. This refinement is essential for stable adversarial learning in complex, noisy environments.

Convergence and Stability

Early adversarial networks often struggled with training stability. However, the paper notes that modern mathematical proofs, specifically those utilizing two-time scale stochastic approximation, have validated the convergence of these systems to stable equilibria. While these proofs do not guarantee a global optimum—a challenge that remains in modern GAN research—they provide a rigorous foundation for the minimax training paradigm.

Conclusion

The success of GANs is a testament to the power of adversarial learning, but it is also a continuation of a research thread that began decades ago. By recognizing GANs as a form of "blind" Artificial Curiosity, we gain a more holistic view of unsupervised learning. This perspective suggests that the techniques used to drive exploration in reinforcement learning, such as learning progress rewards, may hold untapped potential for improving the efficiency and stability of generative models today.

Summary updated

BibTeX
@article{schmidhuber2020unsupervised,
  author = {Jürgen Schmidhuber},
  title = {{Unsupervised Minimax: Adversarial Curiosity, Generative Adversarial Networks, and Predictability Minimization}},
  journal = {Neural Networks},
  year = {2020},
  doi = {10.1016/j.neunet.2020.04.008},
}
Figure