Choosing and learning: outcome valence differentially affects learning from free versus forced choices

Valérian Chambon, Héloïse Théro, Marie Vidal, Henri Vandendriessche, Patrick Haggard, Stefano PalminteribioRxiv 2019 · DOI

TL;DR

Humans don't learn equally from good and bad news; they tend to favor "good news" (positivity bias). This paper shows this bias isn't about positive events themselves, but rather a "choice-confirmation" bias: we learn more from information that supports our own decisions. This bias is adaptive, helping us reinforce actions that meet our needs when we have control, but disappears when choices are imposed.

What Problem Does This Paper Address?

We often hear that people have a "positivity bias" – they learn more from positive outcomes than negative ones. This has been interpreted as a general preference for good news. However, this paper questions whether this bias is truly about the valence (positive/negative) of the outcome itself, or if it's a consequence of the act of choosing. When we make a decision, we tend to seek out and integrate information that confirms our choice, a phenomenon known as "confirmation bias." The authors propose that the observed "positivity bias" might actually be a specific instance of this broader "choice-confirmation" bias. They aim to disentangle these two explanations by comparing learning in situations where people freely choose between options versus situations where their choice is dictated by a computer.

Methods

The researchers conducted three experiments using a probabilistic instrumental learning task. Participants had to learn which of two symbols offered a better reward probability.

  • Experiments 1 & 2: These experiments intermixed "free-choice" trials (participants chose freely) and "forced-choice" trials (participants had to match a computer-selected option). Experiment 1 used "partial feedback" (only the outcome of the chosen option was shown), while Experiment 2 used "complete feedback" (outcomes for both chosen and unchosen options were shown).
  • Experiment 3: This experiment focused on "free-choice" trials but varied the motor requirements: participants either had to press a key ("go" trials) or refrain from pressing a key ("no-go" trials) to make their choice.

Crucially, they employed computational modeling, specifically a modified Rescorla-Wagner learning model, to analyze participants' learning rates. They fitted different learning rates for positive vs. negative outcomes and for free vs. forced choices, and in Experiment 2, for obtained vs. forgone outcomes. Model comparison techniques (Bayesian Information Criterion - BIC) were used to determine which model best explained the data.

Datasets

Not discussed in the paper. The study used data collected from human participants performing the described tasks.

Key Insights and Ideas

  • Positivity Bias is a Choice-Confirmation Bias: The most significant finding is that the tendency to learn more from positive outcomes is not a general preference for good news, but rather a bias that emerges only when individuals have made a free choice. When choices are imposed, this bias disappears.
  • Confirmation Bias Operates Early: The bias appears to stem from the decision-making stage itself, not from the physical execution of the action. Whether participants pressed a button or not (go vs. no-go trials) didn't change the learning asymmetry, suggesting the "choice" itself is the key.
  • Adaptive Nature of Choice-Confirmation: The authors propose this bias is adaptive. By reinforcing our own choices, we prioritize information that aligns with our needs and goals, which is beneficial when we have agency.
  • Impartial Learning for Imposed Choices: When choices are forced, learning becomes valence-neutral. This makes sense, as imposed outcomes don't necessarily reflect our own needs or values, so treating them impartially is more appropriate.
  • Counterfactual Learning Reverses Bias: Interestingly, when learning from unchosen actions (counterfactual learning), the bias flips: negative prediction errors (outcomes that contradict the unchosen action) were better integrated than positive ones, further supporting the choice-confirmation idea.

Key Results

  • Learning Asymmetry in Free Choice: Participants showed a significant learning asymmetry, learning more from positive prediction errors than negative ones, but only in free-choice trials.
  • No Asymmetry in Forced Choice: This valence-induced learning asymmetry was absent in forced-choice trials, where learning was valence-neutral.
  • Action Execution Irrelevant: In Experiment 3, varying the motor action required for a choice (go vs. no-go) did not affect the learning asymmetry, reinforcing the idea that the bias is tied to the act of choosing, not the physical action.
  • Model Comparison Favors Choice-Confirmation: Computational models strongly supported the choice-confirmation bias theory over a simple positivity bias. The winning model had different learning rates for positive/negative outcomes in free choices, but uniform rates in forced choices.

Limitations and Future Work

  • Generalizability: While the study used three experiments and computational modeling, future work could explore this bias in more complex, real-world decision-making scenarios.
  • Individual Differences: The paper touches on individual differences in controllability beliefs, but further research could delve deeper into how personality traits or prior experiences influence the strength of this choice-confirmation bias.
  • Optimality in Different Environments: The authors discuss the adaptive nature of this bias, particularly in volatile environments. Future work could explore how this bias performs in environments with different reward structures and contingencies.
  • Perseveration Bias: The authors addressed the possibility that their findings could be explained by a simple choice repetition bias (perseveration). While their models favored the choice-confirmation account, they acknowledge that learning-independent choice perseveration might play a role in specific contexts.

Summary updated

Abstract

Abstract Positivity bias refers to learning more from positive than negative events. This learning asymmetry could either reflect a preference for positive events in general, or be the upshot of a more general, and perhaps, ubiquitous, “choice-confirmation” bias, whereby agents preferentially integrate information that confirms their previous decision. We systematically compared these two theories with 3 experiments mixing free- and forced-choice conditions, featuring factual and counterfactual learning and varying action requirements across “go” and “no-go” trials. Computational analyses of learning rates showed clear and robust evidence in favour of the “choice-confirmation” theory: participants amplified positive prediction errors in free-choice conditions while being valence-neutral on forced-choice conditions. We suggest that a choice-confirmation bias is adaptive to the extent that it reinforces actions that are most likely to meet an individual’s needs, i.e. freely chosen actions. In contrast, outcomes from unchosen actions are more likely to be treated impartially, i.e. to be assigned no special value in self-determined decisions.
BibTeX
@misc{chambon2019choosing,
  author = {Valérian Chambon and Héloïse Théro and Marie Vidal and Henri Vandendriessche and Patrick Haggard and Stefano Palminteri},
  title = {{Choosing and learning: outcome valence differentially affects learning from free versus forced choices}},
  year = {2019},
  doi = {10.1101/637157},
  howpublished = {bioRxiv},
}
Figure