TL;DR
This paper reveals that our brains learn differently from what actually happened versus what could have happened. While we tend to focus on positive outcomes when learning from our own choices (factual learning), we actually pay more attention to negative outcomes when considering what we missed out on (counterfactual learning). This suggests a "confirmation bias" where we prioritize information that validates our past decisions.
What Problem Does This Paper Address?
We constantly make decisions and learn from their consequences. This learning process is crucial for adapting our behavior. Previous research has shown that when learning from the outcomes of our chosen actions (factual learning), people exhibit a "valence bias," meaning they learn more from positive outcomes than negative ones. This is often framed as an "optimistic bias." However, a significant question remained: does this valence bias also apply to learning from forgone outcomes (counterfactual learning) – what we could have experienced if we had made a different choice? Understanding this is key because much of our decision-making involves considering alternatives.
Methods
The researchers designed two experiments using a reinforcement learning task. Participants had to choose between two options, and the outcomes were either winning or losing points.
- Experiment 1: Participants only saw the outcome of their chosen option, allowing for the study of factual learning.
- Experiment 2: Participants saw the outcomes of both their chosen and unchosen options, enabling the study of both factual and counterfactual learning.
They employed a computational model, a modified Rescorla-Wagner model, to analyze the learning rates. This model allowed them to estimate separate learning rates for positive and negative prediction errors, and for both factual (chosen) and counterfactual (unchosen) outcomes. Model comparison was used to determine the most parsimonious explanation for the observed learning patterns.
Datasets
Not discussed in the paper. The study involved two cohorts of human participants.
Key Insights and Ideas
- Opposing Biases: The most striking finding is that factual and counterfactual learning are biased in opposite directions. This challenges the idea of a single, overarching "positive valence bias."
- Confirmation Bias: The results strongly support a "confirmation bias" hypothesis. This means we preferentially learn from outcomes that confirm our previous choices, regardless of whether they were factual or counterfactual. Positive outcomes from chosen options and negative outcomes from unchosen options are weighted more heavily.
- Adaptive or Maladaptive? The study suggests these biases might be maladaptive, particularly in situations requiring rapid adaptation to changing reward contingencies (like in the "Reversal" condition). Participants with stronger biases performed worse in these scenarios.
- Unified Learning System: The model comparison results suggest that factual and counterfactual learning might be processed by the same underlying learning system, rather than separate ones.
Key Results
- Factual Learning: Replicated previous findings, showing a "positive valence bias" where participants learned more from positive prediction errors than negative ones (e.g., learning more from winning a point than losing one when it was the chosen outcome).
- Counterfactual Learning: Revealed an opposite bias. Participants learned more from negative prediction errors than positive ones when considering forgone outcomes (e.g., paying more attention to the fact that they could have won a point but didn't, compared to the fact that they could have lost a point and didn't).
- Model Comparison: The "Confirmation" model, which accounts for biases based on whether an outcome confirms or contradicts a choice, provided a better fit to the data than models focusing solely on valence or outcome type.
- Behavioral Impact: Participants with higher learning biases showed a tendency to stick with a preferred choice even when there was no objectively "correct" option and performed worse in conditions requiring adaptation to changing reward probabilities.
Limitations and Future Work
- Maladaptive Nature: While the biases appear maladaptive in certain contexts, the authors speculate they might have an overall adaptive value by contributing to self-esteem and confidence (e.g., "self-serving" biases). Further research could explore this trade-off.
- Neurobiological Basis: The study provides behavioral and computational evidence. Future work could investigate the specific neural mechanisms underlying these opposing biases in factual and counterfactual learning.
- Generalizability: The study used specific task conditions. Exploring these biases in more complex, real-world decision-making scenarios would be valuable.
- Explicit vs. Implicit Confirmation: The authors note that their definition of confirmation bias extends to implicitly reinforcing current choices, which differs from previous work focusing on explicit advice or instructions. Further research could delineate these different forms of confirmation bias.
Summary updated