TL;DR
This study investigates how the valence of outcomes (gains vs. losses) influences human decision-making speed and confidence in reinforcement learning. The researchers found that while losses consistently slow down responses and decrease confidence, the confidence bias can persist even when the slowing of response times is experimentally reduced, suggesting these two biases are partially dissociable.
What Problem Does This Paper Address?
In reinforcement learning, humans are generally adept at learning to both seek rewards and avoid punishments. However, research has shown that the valence of these outcomes can lead to asymmetric effects on behavior. Specifically, decisions involving potential losses tend to be slower (a "motor bias") and participants report lower confidence in their choices (a "confidence bias") compared to decisions involving potential gains. The core question this paper addresses is whether these two biases are driven by a single underlying mechanism or if they can be separated. Understanding this dissociation is crucial for developing more accurate computational models of human decision-making and metacognition.
Methods
The researchers conducted six experiments using variations of a probabilistic instrumental learning task. Participants learned to associate abstract symbols with monetary outcomes (gains or losses) and had to choose the symbol that maximized their potential payoff.
Key methodological innovations included:
- Manipulating the option-action mapping: In some experiments, the mapping between the choice (e.g., left vs. right symbol) and the physical response (e.g., left vs. right button) was altered or delayed. This was done to disrupt the direct link between the decision and the motor execution, aiming to reduce the valence-induced slowing of response times (RTs).
- Imposing time pressure: One experiment introduced strict time constraints on decision-making to further limit response times.
- Measuring confidence: Participants reported their confidence in their choices on a graded scale, often incentivized using a matching probability procedure to encourage truthful reporting.
- Statistical analysis: The study employed repeated-measures ANOVAs and generalized linear mixed-effect models (GLME) to analyze the data at both the individual experiment and overall population levels. They also used inter-individual regression analyses to explore the relationship between the magnitude of the RT bias and the confidence bias across participants.
Datasets
The study utilized data from six experiments. Two of these datasets were previously published (Experiments 1 and 2), and four were newly collected (Experiments 3-6). The experiments were adapted from a standard probabilistic instrumental learning paradigm. No new benchmark datasets were introduced; rather, existing paradigms were modified to test specific hypotheses.
Key Insights and Ideas
- Robustness of Valence Biases: The study strongly confirms that both the motor bias (slower RTs in loss contexts) and the confidence bias (lower confidence in loss contexts) are highly robust and replicable across various experimental manipulations.
- Partial Dissociability: The most significant finding is that the confidence bias can persist even when the motor bias is significantly attenuated or absent. This suggests that while often occurring together, these two biases are not entirely dependent on each other.
- Confidence as More Than Just RT: The results challenge the idea that confidence judgments are merely a readout of response times. The persistence of the confidence bias when RTs are normalized suggests that confidence reflects other underlying processes, potentially related to the subjective evaluation of the decision itself, rather than just the speed of execution.
- Implications for Models: The findings highlight the need for computational models of reinforcement learning and metacognition to account for these partially dissociable valence effects, moving beyond simple, unified mechanisms.
Key Results
- Across all six experiments, participants consistently showed slower response times (RTs) in loss contexts compared to gain contexts. This motor bias was remarkably resilient to experimental manipulations designed to disrupt it.
- Similarly, participants consistently reported lower confidence in their choices in loss contexts compared to gain contexts. This confidence bias was also highly replicable.
- In Experiment 5, the researchers were able to significantly reduce or eliminate the valence-induced RT bias. Crucially, even in this condition, the valence-induced confidence bias remained significant.
- Inter-individual analyses revealed that participants who exhibited a stronger valence-induced RT bias also tended to show a stronger valence-induced confidence bias, indicating a link, but not a complete dependency. The intercept of these regressions showed that even individuals with no RT bias still exhibited a confidence bias.
Limitations and Future Work
- Experimental Manipulations: While the researchers attempted to disrupt the motor bias, they acknowledge that it's possible the experimental setups were not entirely successful in canceling it, and the null effect in Experiment 5 could be a "false negative."
- Neurobiological Basis: The paper speculates about the neural systems involved (e.g., ventral striatum, vmPFC) but emphasizes that these are neuroanatomical hypotheses requiring future neuroimaging studies for confirmation.
- Computational Modeling: The authors suggest that future work should refine computational models to incorporate these partially dissociable valence effects. They also note that their current experimental designs do not allow for the application of combined reinforcement-learning and decision-diffusion modeling approaches used in prior work.
- Decision Time vs. Response Time: The paper distinguishes between decision time (when a decision is made) and response time (when the choice is executed). Future research could further investigate how internal measures of decision time might influence confidence, especially in light of the observed dissociation.
Summary updated