Abstract
Human function learning has long been debated through two competing lenses: rule-based estimation and similarity-based associative learning. This paper proposes a unifying rational model based on Gaussian Processes (GPs), demonstrating that these two perspectives are mathematically dual. By employing a mixture of kernels to represent human inductive biases, the model successfully accounts for interpolation difficulty, extrapolation patterns, and complex phenomena like knowledge partitioning and iterated learning.
Introduction
How do humans learn the relationship between continuous variables? Cognitive psychology has traditionally been split between rule-based theories, where individuals estimate explicit functions (e.g., $y = mx + b$), and similarity-based theories, where associative learning links similar inputs to similar outputs. While both explain specific behaviors, neither fully accounts for the breadth of human performance, such as the strong bias toward linear extrapolation or the ability to "partition" knowledge across different regions of an input space. This work seeks a "rational analysis" to unify these perspectives by identifying the optimal solution to regression given human-like constraints.
Theoretical Framework: The Duality of Rules and Similarity
The core insight of this research is the mathematical equivalence between Bayesian linear regression (rules) and Gaussian processes (similarity). Any rule-based model using basis functions has an equivalent GP kernel, and conversely, any kernel can be viewed as an infinite set of rules.
To capture human behavior, the authors propose a model that treats function learning as Bayesian inference over a mixture of kernels:
- Model 1: A mixture of four kernels: Positive Linear, Negative Linear, Quadratic, and Nonlinear (Radial Basis Function).
- Model 2: Incorporates a Chinese Restaurant Process (CRP) prior to allow for learning multiple, separate relationships simultaneously.
- Model 3 (Mixture of Experts): Refines the approach by allowing "experts" to be local, enabling the model to apply different rules to different regions of the input space.
Experimental Results
The model was evaluated against empirical phenomena from classic psychology studies, including interpolation difficulty, extrapolation patterns, and knowledge partitioning.
Knowledge Partitioning
A key challenge for unimodal models is "knowledge partitioning," where humans learn discontinuous or bimodal relationships. As shown in Figure 1, the model (specifically Model 3) successfully replicates the response distributions of individual participants when faced with discontinuous training data, capturing the multimodality that previous models could not.
Iterated Learning and Inductive Biases
In iterated learning tasks, where functional beliefs are passed through generations (similar to a game of "telephone"), human populations tend to converge toward linear functions. The GP model accurately mirrors this convergence, reflecting the "rational" prior that the world is usually simple and linear. Figure 3 illustrates how GP Model 3 compares to other cognitive models (EXAM and POLE) in simulating these evolutionary chains across different starting conditions.
Performance and Extrapolation
The GP model demonstrated high correlation ($r \approx 0.93$) with human error rates across various function types, correctly predicting that linear functions are easiest to learn, followed by quadratic and cyclic forms. Furthermore, the mixture-of-kernels approach captured the human tendency to "linearize" non-linear functions during extrapolation, a feat that pure similarity or pure rule models often fail to achieve.
Discussion
This research suggests that the "rules vs. similarity" debate may be a category error; they are simply different descriptions of the same underlying Bayesian inference. Human biases, such as the preference for linearity, are modeled as prior probabilities over specific kernels. While the model currently assumes perfect memory and struggles with complex cyclic functions, it provides a robust computational-level explanation of how humans approximate optimal Bayesian regression. Future work will focus on integrating periodic kernels and bridging these findings to algorithmic-level neural implementations.
Summary updated