Dissecting the links between reward and loss, decision-making, and self-reported affect using a computational approach

Vikki Neville, Peter Dayan, Iain Donald Gilchrist, Elizabeth S. Paul, Michael Mendl

TL;DR

This study reveals how short-term experiences of rewards and punishments within a decision-making task influence both our choices and our emotional states. By developing a novel computational model, the researchers show that unpredictability in outcomes leads to more cautious (risk-averse) decisions, and that our feelings are directly shaped by the immediate feedback we receive.

What Problem Does This Paper Address?

Traditionally, researchers have used summary statistics from decision-making tasks to understand the links between affect (emotions/mood) and behaviors like risk-taking. However, this approach often overlooks a crucial aspect: how the ongoing experience of rewards and punishments during the task itself can dynamically alter both future decisions and our emotional state. This paper tackles the gap in understanding this immediate, within-task interplay between affect, reward/punishment experiences, and decision-making. They aim to show that decision-making isn't just a static reflection of a person's general mood, but is actively shaped by the immediate consequences of their actions.

Methods

The researchers employed a novel computational approach based on a Partially Observable Markov Decision Process (POMDP). This is a sophisticated framework for modeling sequential decision-making under uncertainty. They adapted a standard "judgement bias task" (often used in animal studies to assess affect) for human participants. In this task, participants had to classify the direction of motion in a random dot display, with different directions associated with monetary rewards or losses.

The key novelty lies in their computational model, which went beyond simply analyzing choice patterns. It aimed to:

  • Model the dynamics of decision-making: Capturing how information is accumulated over time and how this influences choices.
  • Quantify within-task experience: Extracting key statistics like average earning rate, prediction errors (the difference between expected and actual outcomes), and the unpredictability of outcomes (squared prediction error).
  • Link experience to decision parameters: Investigating how these experiential factors modulated core decision-making parameters like loss sensitivity and prior beliefs.
  • Connect decisions to affect: Relating the model's estimated parameters and the task experiences to self-reported affective states (valence and arousal) collected throughout the experiment.

Datasets

The study used data collected from human participants performing a custom-designed judgement bias task. No external datasets were used for training or evaluation. The data collected included:

  • Choice data: Whether participants made a "stay" (optimistic) or "go" (pessimistic) response.
  • Reaction time data: How long it took to make a decision.
  • Self-reported affect data: Participants used a computerised affect grid to report their current mood (valence and arousal) at various points during the task.

Key Insights and Ideas

  • Unpredictability breeds caution: When outcomes in the task were more unpredictable (higher squared prediction error), participants became more risk-averse, opting for the "pessimistic" (loss-avoiding) choice. This suggests that uncertainty directly influences our willingness to take risks.
  • Losses loom larger: The model revealed that participants' sensitivity to losses, rather than rewards or prior beliefs, was a stronger driver of their decision-making, especially when outcomes were unpredictable. This aligns with the well-established notion that losses are more salient than gains.
  • Affect is a real-time feedback loop: The study provides strong evidence that immediate experiences within a task directly shape our emotional state. Higher average earnings, more positive prediction errors, and more positive outcomes were all associated with more positive affective valence.
  • Affect influences decision-making, but not always directly: While affect and decision-making are linked, the study suggests that the influence isn't always a simple mediation. For instance, average earning rate influenced affect but not directly decision-making bias, while unpredictability influenced decision-making but not directly affect. This highlights a complex, potentially dissociable relationship.

Key Results

  • Model Fit: The developed POMDP model provided a good fit to the choice data, capturing individual differences and trial-by-trial variations.
  • Unpredictability and Risk Aversion: Greater unpredictability of recent outcomes (squared prediction error) led to increased loss valuation and more "pessimistic" (risk-averse) decisions.
  • Affective Valence and Task Experience: Participants reported more positive affective valence when they experienced higher average earning rates, more positive prediction errors, and more positive outcomes.
  • Affective Valence and Loss Valuation: Greater valuation of losses was associated with more positive reported affective valence in the fluctuating reward condition.
  • Affective Arousal: Greater arousal was reported when offered rewards were higher, suggesting increased alertness for high-stakes trials.

Limitations and Future Work

  • Short Timescale: The study focused on short-term, within-task experiences. The authors acknowledge that longer-term mood might play a more significant role in decision-making over extended periods.
  • Task Specificity: The findings regarding the influence of unpredictability on loss aversion were primarily reliable in the fluctuating reward condition. This might be due to the task's design where loss avoidance was guaranteed by the "go" response, but reward gain was not.
  • Model Complexity: Some parameters in the model did not significantly improve parsimony and might require larger datasets or different fluctuation patterns to be detected.
  • Future Directions: The authors suggest exploring these relationships in clinical populations to understand aberrant decision-making. They also propose investigating the neurobiological underpinnings of the model parameters and exploring how pharmacological manipulations of affect might influence judgement bias. Further research is needed to understand the interplay between longer-term mood and environmental conditions on affective states.

Summary updated

Abstract

Links between affective states and risk-taking are often characterised using summary statistics from serial decision-making tasks. However, our understanding of these links, and the utility of decision-making as a marker of affect, needs to accommodate the fact that ongoing (e.g. within-task) experience of rewarding and punishing decision outcomes may alter future decisions and affective states. To date, the interplay between affect, ongoing reward and punisher experience, and decision-making has received little detailed investigation. Here, we examined the relationships between reward and loss experience, affect, and decision-making in humans using a novel judgement bias task analysed with a novel computational model. We demonstrated the influence of within-task favourability on decision-making, with more risk-averse/`pessimistic' decisions following more positive previous outcomes and a greater current average earning rate. Additionally, individuals reporting more negative affect tended to exhibit greater risk-seeking decision-making, and, based on our model, estimated time more poorly. We also found that individuals reported more positive affective valence during periods of the task when prediction errors and offered decision outcomes were more positive. Our results thus provide new evidence that (short-term) within-task rewarding and punishing experiences determine both future decision-making and subjectively experienced affective states.
BibTeX
@article{neville2021dissecting,
  author = {Vikki Neville and Peter Dayan and Iain Donald Gilchrist and Elizabeth S. Paul and Michael Mendl},
  title = {{Dissecting the links between reward and loss, decision-making, and self-reported affect using a computational approach}},
  journal = {PLOS Computational Biology},
  year = {2021},
  doi = {10.1371/journal.pcbi.1008555},
}
Figure