Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes

Jürgen Schmidhuber

TL;DR

This paper proposes a unified theory that explains a wide range of cognitive phenomena, from subjective beauty and curiosity to creativity and scientific discovery, as emergent properties of a single principle: the drive to compress data. The authors argue that systems are intrinsically motivated to find patterns and regularities in data because doing so makes the data simpler, more predictable, and thus "beautiful" and "interesting."

What Problem Does This Paper Address?

The paper tackles the fundamental question of what drives intelligent behavior, particularly in the absence of external rewards. While traditional AI often focuses on optimizing for explicit goals (like maximizing points in a game), this work seeks to explain intrinsic motivations like curiosity, creativity, and the appreciation of beauty. The authors argue that these seemingly disparate phenomena can be unified under a single, powerful principle rooted in data compression. They aim to bridge the gap between computational theories of intelligence and subjective human experiences.

Methods

The core of the paper's approach is an algorithmic framework based on three key principles:

  1. Store Everything: Agents should retain all sensory input and actions as the basis for learning.
  2. Improve Subjective Compressibility: Agents should actively try to compress their data history using adaptive algorithms. This means finding patterns and regularities that allow for a shorter representation of the data.
  3. Reward Compression Progress: Intrinsic reward (curiosity) should be generated in proportion to the improvement in data compression. This means the agent is rewarded not just for achieving good compression, but for getting better at compressing the data.
  4. Maximize Intrinsic Curiosity Reward: A reinforcement learning (RL) agent uses this intrinsic reward to guide its exploration and action selection, aiming to discover new, previously unknown regularities.

The authors propose that this framework, when implemented, can naturally lead to phenomena like attention, novelty detection, surprise, and ultimately, creativity and aesthetic appreciation.

Datasets

Not discussed in the paper. The paper focuses on a theoretical framework and algorithmic principles, illustrated with conceptual examples rather than empirical datasets.

Key Insights and Ideas

  • Beauty as Compressibility: Subjective beauty is defined as the simplicity of an observation given an observer's current knowledge. A more beautiful object is one that is more easily compressed.
  • Interestingness as the Derivative of Beauty: Interestingness is not static beauty, but the rate at which beauty improves – the steepness of the learning curve as an agent discovers new regularities. This explains why novelty is engaging.
  • Curiosity as a Drive for Compression Progress: The intrinsic motivation to explore and learn is framed as a drive to maximize the rate of compression progress. This explains why agents (and humans) seek out novel and challenging information.
  • Art and Science as By-products: Creativity in art and science is seen as the process of discovering or creating data with novel, compressible regularities. Science formalizes these discoveries, while art often embodies them.
  • Rejection of Traditional Surprise: The paper argues against traditional information-theoretic definitions of surprise (based on low probability). Instead, "true" surprise comes from data that is currently incompressible but is expected to become compressible with further learning.

Key Results

The paper presents a theoretical framework and conceptual arguments rather than empirical results with quantitative metrics. The "results" are the explanations provided for various cognitive phenomena:

  • Explanation of Subjective Phenomena: The framework offers a unified explanation for why humans find certain things beautiful, interesting, or surprising, and why they are driven by curiosity and creativity.
  • Algorithmic Principles: The paper outlines concrete algorithmic principles that can be implemented in artificial systems to exhibit these behaviors.
  • Visual Examples: Figures are provided to illustrate how simple geometric patterns can lead to "beautiful" and compressible images, demonstrating the core idea visually.

Limitations and Future Work

  • Incomputability of Optimal Solutions: The paper acknowledges that truly optimal predictors and action selectors (like Solomonoff's universal predictor or AIXI) are incomputable. Practical implementations must rely on approximations and heuristics.
  • Computational Cost: Evaluating compressor performance and improvement can be computationally expensive, especially on long histories.
  • Future Work: The authors suggest several avenues for future research:
  • Studying better adaptive compressors (e.g., novel RNNs).
  • Investigating efficient ways to compute learning progress measures.
  • Applying advanced RL techniques (policy gradients, evolutionary methods) to the curiosity framework.
  • Testing the theory through controlled psychological experiments and neurophysiological studies.
  • Implementing the principles on robots.

Summary updated

BibTeX
@inproceedings{schmidhuber2009driven,
  author = {Jürgen Schmidhuber},
  title = {{Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes}},
  booktitle = {Computational Creativity: An Interdisciplinary Approach},
  year = {2009},
}

Loading related content...

Figure