Bayesian inference for psychometric functions

Malte Kuß, Frank Jäkel, Felix A. WichmannJournal of Vision 2005 · DOI

TL;DR

This paper introduces a Bayesian approach using Markov Chain Monte Carlo (MCMC) for estimating psychometric functions, which model the relationship between stimulus intensity and an observer's response. The authors demonstrate that this Bayesian method yields more accurate point estimates and confidence intervals compared to traditional maximum likelihood estimation with bootstrapping, offering a more robust way to analyze psychophysical data.

What Problem Does This Paper Address?

Psychophysical studies often rely on psychometric functions to understand how sensitive observers are to changes in stimulus intensity. Traditionally, researchers focus on estimating a single point (like the "threshold" at 50% correct response) or use methods like maximum likelihood estimation (MLE) with bootstrapping to get confidence intervals. However, these methods can be less accurate, especially with limited data or when considering the entire function. The authors highlight that a comprehensive understanding of the psychometric function, including its shape and variability, is crucial for many applications, and existing methods struggle to provide reliable estimates and confidence intervals for all parameters, particularly the "lapse rate" (probability of random guessing). They aim to provide a more principled and accurate framework for this estimation.

Methods

The core of the paper is the application of Bayesian inference to psychometric functions. Since analytical solutions for the posterior distribution are intractable, the authors employ Markov Chain Monte Carlo (MCMC), specifically hybrid Monte Carlo sampling (also known as Hamiltonian sampling), to generate samples from the posterior distribution of the psychometric function's parameters.

Here's a breakdown:

  • Binomial Mixture Model: They start with a standard model for psychometric data, a binomial mixture model that accounts for stimulus intensity, correct response probability, and importantly, a "lapse rate" (probability of guessing) and "chance probability" (related to the number of response options).
  • Bayesian Framework: They frame the problem as inferring the parameters of this model (threshold, width, lapse rate) given observed data. This involves defining prior distributions for these parameters and using Bayes' rule to update beliefs based on the data.
  • MCMC Sampling: Because the posterior distribution cannot be computed directly, MCMC is used to draw samples that approximate this distribution. These samples are then used to estimate point estimates (mean, median, MAP) and confidence intervals.
  • Parameterization: They discuss and advocate for a specific parameterization of psychometric functions (using threshold m and width w) that is more intuitive and facilitates Bayesian analysis.

Datasets

The paper uses two types of data:

  • Artificially Generated Data: The authors generate synthetic datasets from a known Gumbel psychometric function with specified parameters to test their method under controlled conditions. This allows for direct comparison of estimated parameters against the true generating values.
  • Real Experimental Data: They analyze data from a visual contrast-discrimination task published by Henning, Bird, and Wichmann (2002). This case study involves two conditions (sine wave and pulse train) and provides a real-world test of their Bayesian approach.

Key Insights and Ideas

  • Bayesian Inference Superiority: The central finding is that Bayesian inference with MCMC provides more accurate point estimates and more reliable (tighter and more accurate) confidence intervals than traditional bootstrap methods, especially with limited data.
  • Importance of Priors: The paper emphasizes that prior distributions are crucial and can significantly influence the posterior, particularly for parameters like the lapse rate where data might be less informative. They highlight that "flat" priors on parameters don't necessarily mean "uninformative" in terms of the psychometric function's shape.
  • MCMC for Complex Models: MCMC is presented as a powerful tool for handling complex statistical models where analytical solutions are impossible, enabling principled inference.
  • Understanding Uncertainty: The MCMC samples provide a rich representation of uncertainty, allowing for deeper insights into the relationships between parameters (e.g., the negative correlation between lapse rate and width).
  • Parameterization Matters: The choice of how to parameterize the psychometric function (e.g., using threshold and width) can significantly impact the interpretability and behavior of the prior and posterior distributions.

Key Results

  • Point Estimates: The MCMC-derived median (MED) consistently showed good accuracy for the threshold parameter (m). For the width parameter (w), the MAP estimate was followed by MED, both outperforming ML.
  • Confidence Intervals: Bayesian confidence intervals were found to be more accurate and generally tighter than bootstrap intervals. For the threshold (m), Bayesian intervals were close to the desired 90% accuracy, while bootstrap intervals were often over-confident. For the width (w), both methods were relatively accurate, but Bayesian intervals tended to be more conservative and narrower.
  • Lapse Rate Influence: The lapse rate parameter was found to be difficult to estimate accurately, especially with smaller datasets, and its prior had a substantial influence on the posterior.
  • Case Study Findings: In the real data analysis, the Bayesian approach confirmed findings from previous studies, showing no substantial difference in discrimination thresholds between sine wave and pulse train stimuli, despite the pulse train having additional frequency components.

Limitations and Future Work

  • MCMC Complexity: The authors acknowledge that using MCMC is nontrivial and requires expertise in setting parameters (like step sizes and number of steps) and inspecting the chains for convergence. They mention that this often involves a trial-and-error process.
  • Prior Sensitivity: While the Bayesian approach is powerful, its sensitivity to prior choices, especially for parameters like the lapse rate, is a significant consideration. They suggest that reporting both prior and posterior distributions is essential.
  • Software Availability: To address the complexity, they mention releasing a software implementation package called PsychoFun for the R environment, which includes guidance on implementation and usage.
  • Generalizability: While they found their Bayesian method superior in their experiments, they acknowledge that proving this generalization to all possible data situations is not possible, though they see no reason for it to perform worse.
  • Future Work: The paper implicitly suggests that further exploration of different prior choices, more complex psychometric function models, and applications to various psychophysical tasks would be valuable. The development of user-friendly software like PsychoFun is also a step towards broader adoption.

Summary updated

Abstract

In psychophysical studies, the psychometric function is used to model the relation between physical stimulus intensity and the observer's ability to detect or discriminate between stimuli of different intensities. In this study, we propose the use of Bayesian inference to extract the information contained in experimental data to estimate the parameters of psychometric functions. Because Bayesian inference cannot be performed analytically, we describe how a Markov chain Monte Carlo method can be used to generate samples from the posterior distribution over parameters. These samples are used to estimate Bayesian confidence intervals and other characteristics of the posterior distribution. In addition, we discuss the parameterization of psychometric functions and the role of prior distributions in the analysis. The proposed approach is exemplified using artificially generated data and in a case study for real experimental data. Furthermore, we compare our approach with traditional methods based on maximum likelihood parameter estimation combined with bootstrap techniques for confidence interval estimation and find the Bayesian approach to be superior.
BibTeX
@article{ku2005bayesian,
  author = {Malte Kuß and Frank Jäkel and Felix A. Wichmann},
  title = {{Bayesian inference for psychometric functions}},
  journal = {Journal of Vision},
  year = {2005},
  doi = {10.1167/5.5.8},
}
Figure