Modeling human function learning with Gaussian processes

Thomas L. Griffiths, Christopher G. Lucas, Joseph Jay Williams, Michael L. KalishNeurIPS 2008 · OpenReview

Overview

This paper unifies the two dominant—and previously conflicting—theories of how humans learn functional relationships: explicit rules versus associative similarity. By framing these theories through the lens of Gaussian Processes (GPs), the authors demonstrate that "rules" and "similarity" are mathematically equivalent views of the same regression problem. This insight allows for a single, rational model that captures human-like learning and extrapolation without relying on ad-hoc hybrid mechanisms.

The Problem: How Do We Learn Functions?

When humans learn the relationship between two continuous variables (e.g., how the pressure on a gas pedal affects a car's speed), how do we internalize and predict this relationship? Historically, cognitive science has been split into two camps:

  1. Rule-based models: These assume we fit data to a specific functional form, like a linear or quadratic equation, and estimate the parameters. This explains our ability to predict values far outside our experience (extrapolation), but it fails to explain how we "pick" the right rule in the first place.
  2. Similarity-based (Associative) models: These suggest we store observations and generalize to new points based on how similar they are to what we’ve already seen. While this explains the learning process well, it fails miserably at extrapolation, often reverting to a local average when moving beyond the training data.

The authors seek to bridge this gap by providing a "rational analysis"—an explanation of human behavior based on optimal statistical solutions to the problem of regression.

Methods: Gaussian Processes as a Unifying Framework

The researchers employ Gaussian Processes (GPs), a non-parametric Bayesian approach to regression. The core of their argument rests on a mathematical duality:

  • Bayesian Linear Regression (representing "rules") uses basis functions to define a hypothesis space.
  • Gaussian Process Prediction (representing "similarity") uses a kernel function to define how much two points influence each other.

The paper shows that any rule-based model using basis functions has an equivalent GP kernel, and any kernel can be viewed as an infinite set of rules. Choosing a kernel is choosing a prior over rules.

To model human behavior, the authors propose a GP model that uses a mixture of kernels. Instead of assuming one fixed rule, the model considers a set of potential kernels (e.g., Positive Linear, Negative Linear, Quadratic, and Nonlinear/RBF) and uses Markov Chain Monte Carlo (MCMC) to infer which "type" of function best explains the observed data. Furthermore, to account for the known human "linear bias," the model assigns higher prior probabilities to linear kernels.

Key Results

The model was evaluated against human performance data from classic psychological studies, specifically Byun (1995) and DeLosh, Busemeyer, & McDaniel (1997).

Difficulty Ranking

The GP model's deviation metrics closely matched the order of difficulty humans face when learning different functions. Both humans and the model found linear functions easiest to learn and cyclic or logistic functions the hardest.

Model performance comparisons across different functional forms

As shown in the performance table, the correlation between human and model performance reached as high as 0.93 for the GP models, outperforming or matching previous hybrid approaches without requiring extensive parameter tuning.

Human-like Extrapolation

In the DeLosh et al. (1997) task, the GP model successfully captured the human tendency to extrapolate near-optimally on linear functions while showing a slight "linear pull" (overestimating or underestimating extremes) on quadratic and exponential functions.

Human and model extrapolation on linear, exponential, and quadratic functions

The extrapolation plots demonstrate how closely the GP model mirrors human behavior. Unlike previous hybrid models that manually "stitched" rules and associations together, the GP approach provides a single, mathematically principled framework that naturally handles both interpolation (similarity) and extrapolation (rules).

Limitations and Future Work

  • Non-Monotonicity: The current model struggles with functions that repeat or change direction drastically (like periodic functions) unless specifically given a periodic kernel.
  • Knowledge Partitioning: Humans sometimes learn different functions for different parts of the input space (e.g., "if $x < 0$ it's linear, if $x > 0$ it's quadratic"). The authors suggest that "mixture of experts" GP models could address this in the future.
  • Computational Complexity: While GPs are mathematically elegant, the human brain likely uses an approximation of these calculations rather than full MCMC. Exploring these "online" approximations is a suggested next step for cognitive modeling.

Summary updated

Abstract

Accounts of how people learn functional relationships between continuous variables have tended to focus on two possibilities: that people are estimating explicit functions, or that they are simply performing associative learning supported by similarity. We provide a rational analysis of function learning, drawing on work on regression in machine learning and statistics. Using the equivalence of Bayesian linear regression and Gaussian processes, we show that learning explicit rules and using similarity can be seen as two views of one solution to this problem. We use this insight to define a Gaussian process model of human function learning that combines the strengths of both approaches.
BibTeX
@inproceedings{griffiths2008modeling,
  author = {Thomas L. Griffiths and Christopher G. Lucas and Joseph Jay Williams and Michael L. Kalish},
  title = {{Modeling human function learning with Gaussian processes}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2008},
  url = {https://openreview.net/forum?id=SJbEhUWdbr},
}
Figure