A Bayesian Framework for Modeling Confidence in Perceptual Decision Making

Koosha Khalvati, Rajesh P. RaoNeurIPS 2015 · OpenReview

Introduction

This paper introduces a normative Bayesian framework using Partially Observable Markov Decision Processes (POMDPs) to model how humans and primates estimate confidence during perceptual tasks. By accounting for the fact that decision-makers often lack information available to experimenters (like exact task difficulty), the model successfully predicts experimental confidence reports and explains the "hard-easy effect"—the tendency to be overconfident in difficult tasks and underconfident in easy ones.

The Challenge of Measuring Confidence

In perceptual decision-making—such as deciding if a cloud of dots is moving left or right—accuracy and confidence are related but distinct concepts. While an experimenter can objectively measure accuracy (how often a subject is right), confidence is the subject's internal "self-assessment."

A major challenge in the field is that confidence and accuracy often diverge. For example, if a subject does not know exactly how hard a specific trial is supposed to be, their internal estimate of success (confidence) will not perfectly track their actual performance (accuracy). Previous models, like drift-diffusion or race models, are largely "descriptive"—they describe the data but do not necessarily explain why a brain would calculate confidence in a specific way. The authors seek a "normative" model: one that defines the mathematically optimal way to behave given uncertainty and rewards.

A Dual-POMDP Approach

The authors utilize Partially Observable Markov Decision Processes (POMDPs). A POMDP is a mathematical framework for making optimal decisions when an agent cannot see the true state of the world (e.g., the true direction of motion) and must rely on noisy observations over time.

The key innovation in this paper is the use of two parallel POMDPs:

  1. The Experimenter’s POMDP: This model "knows" the difficulty (coherence) of the trial. It is used to fit the model parameters to the actual accuracy data observed in experiments.
  2. The Decision Maker’s POMDP: This model shares the same parameters but relies on a different "initial belief." It assumes the subject does not know the trial's difficulty beforehand.

By comparing these two models, the authors can predict the subject's internal confidence based solely on the external accuracy data. The model incorporates Bayesian updates (accumulating evidence), costs for "sampling" information (time/effort), and rewards for correct choices.

Experimental Validation

The model was evaluated against two landmark datasets in neuroscience:

  • Fixed-Duration Task (Primate Data): Monkeys performed a motion discrimination task where they could sometimes choose a "sure option" for a small, guaranteed reward instead of guessing the direction. This "post-decision wagering" serves as a behavioral proxy for confidence.
  • Reaction-Time Task (Human Data): Humans performed a similar motion task but chose when to stop and report their decision, followed by an explicit confidence rating on a visual bar.

Key Findings

Confidence as a Belief State

In this framework, confidence isn't a separate, arbitrary "feeling"; it is mathematically derived from the belief state (the posterior probability) of the POMDP.

Explaining the Hard-Easy Effect

The "aha moment" of the paper is that the gap between accuracy and confidence exists because the subject is missing a piece of the puzzle: the trial's difficulty level. When the model accounts for this information asymmetry, the "hard-easy effect" emerges naturally. The model correctly predicted that subjects would be overconfident on low-coherence (hard) trials and underconfident on high-coherence (easy) trials.

Comparison of predicted confidence and actual probability correct over time.

Matching Experimental Data

The POMDP model’s predictions for when a subject would choose the "sure option" closely matched the actual behavior of monkeys across varying stimulus durations and motion strengths.

Probability of choosing the sure target across varying stimulus durations and motion strengths.

Furthermore, in the reaction-time task, the model's predicted belief states showed the same general trends as the explicit confidence bars reported by human subjects. The model successfully accounts for the "cost of time," balancing the desire for more evidence against the discount factor of delayed rewards.

Normative Superiority

Unlike descriptive models that merely fit curves to data, the POMDP approach suggests that the way we report confidence is an optimal strategy for maximizing rewards under uncertainty. It handles the costs and rewards of actions in a way that traditional drift-diffusion models cannot, providing a more holistic view of the decision-making process.

Limitations and Future Directions

  • Model Complexity: The authors assume the decision-maker has a "nearly perfect" internal model of the environment's physics, which might not always hold true for complex or novel tasks.
  • Static States: The current model assumes the "true state" (e.g., motion direction) doesn't change during a trial. Future work could leverage POMDPs to model "intra-trial" changes, where the environment itself is dynamic.
  • Subject Variability: While the model fits "average" behavior well, the reaction-time data was noted to be noisier. This suggests that individual differences in how humans perceive time or reward might require further tuning in future iterations.

Summary updated

Abstract

The degree of confidence in one's choice or decision is a critical aspect of perceptual decision making. Attempts to quantify a decision maker's confidence by measuring accuracy in a task have yielded limited success because confidence and accuracy are typically not equal. In this paper, we introduce a Bayesian framework to model confidence in perceptual decision making. We show that this model, based on partially observable Markov decision processes (POMDPs), is able to predict confidence of a decision maker based only on the data available to the experimenter. We test our model on two experiments on confidence-based decision making involving the well-known random dots motion discrimination task. In both experiments, we show that our model's predictions closely match experimental data. Additionally, our model is also consistent with other phenomena such as the hard-easy effect in perceptual decision making.
BibTeX
@inproceedings{khalvati2015bayesian,
  author = {Koosha Khalvati and Rajesh P. Rao},
  title = {{A Bayesian Framework for Modeling Confidence in Perceptual Decision Making}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2015},
  url = {https://openreview.net/forum?id=rkZoYd-O-B},
}
Figure