TL;DR
This paper tackles the long-standing ambiguity in defining "surprise" by unifying 16 existing mathematical measures into a single framework. It proposes that surprise isn't a monolithic concept but rather a multifaceted one, with different aspects playing crucial roles in learning, perception, and exploration, suggesting distinct neural mechanisms for each.
What Problem Does This Paper Address?
The concept of "surprise" is fundamental to understanding how humans and animals learn, adapt, and make decisions. However, despite its importance, there's no single, universally agreed-upon mathematical definition of surprise. This lack of consensus makes it difficult to interpret experimental results and build accurate computational models of cognitive processes. The authors aim to resolve this by systematically analyzing and categorizing existing surprise measures, revealing their relationships, and proposing experimental paradigms to differentiate them.
Methods
The authors develop a unified generative model of a "volatile environment" to mathematically define and compare various surprise measures. They categorize these measures into four main types: change-point detection, information gain, prediction, and confidence-correction surprise. Crucially, they design experimental paradigms (thought experiments and proposed actual experiments) to test how these different surprise measures make distinct predictions for learning, exploration, and perception, particularly in scenarios involving confidence and environmental volatility. They also introduce a new "Regularized Shannon Surprise" measure designed to better align with intuitive notions of surprise.
Datasets
The paper references publicly available datasets for experimental validation, specifically mentioning data from Robbins et al. (2018) for EEG analysis in an oddball task.
Key Insights and Ideas
- Surprise is not one-size-fits-all: The paper strongly argues against a single mathematical definition of surprise, proposing instead that different aspects of surprise serve distinct computational roles.
- Categorization is key: Unifying and categorizing surprise measures into "prediction," "change-point detection," "information gain," and "confidence-correction" provides a structured way to understand their theoretical underpinnings and experimental implications.
- Confidence matters: Surprise perception is not solely about prediction error; an agent's confidence in its belief significantly influences how surprising an event is, especially in scenarios involving correct vs. incorrect predictions.
- Learning vs. Exploration trade-off: Measures suitable for adaptive learning (change-point detection) are not necessarily optimal for exploration (information gain), and vice-versa, highlighting a fundamental tension in how surprise might be utilized by the brain.
- Bridging the gap to natural language: The authors propose a "Regularized Shannon Surprise" that attempts to better capture the intuitive, everyday meaning of surprise by incorporating confidence in a more nuanced way.
Key Results
The paper demonstrates that:
- Different surprise measures make distinct predictions in various experimental settings, allowing for their empirical dissociation.
- Change-point detection surprise measures are crucial for adaptive learning in volatile environments.
- Information gain surprise measures are more aligned with optimal exploration strategies.
- Confidence plays a significant role, influencing surprise differently depending on whether a prediction was correct or incorrect.
- Existing measures like Shannon and Bayesian surprise do not fully capture intuitive aspects of surprise, particularly when confidence is involved.
Limitations and Future Work
The authors acknowledge that their proposed "Regularized Shannon Surprise" is a step towards a more intuitive definition but still requires further experimental validation. They emphasize the need for future experimental studies to test the proposed paradigms to dissociate the contributions of different surprise measures to biological and behavioral phenomena. They also suggest exploring other surprise theories and their computational roles.
Summary updated