TL;DR
This study reveals that as individuals transition from adolescence to adulthood, their reinforcement learning becomes more "confirmatory" – they learn more from outcomes that align with their expectations. This shift is primarily driven by a reduction in "noise" or exploration, rather than a change in the strength of the confirmation bias itself, offering insights into how learning adapts across the lifespan.
What Problem Does This Paper Address?
Developmental science has long been interested in how learning abilities change throughout human development. Recent computational models have shown that humans exhibit a "confirmatory learning" bias: they learn more from outcomes that confirm their prior expectations than from those that disconfirm them. However, it was unclear whether this bias is stable or if it changes during adolescence, a period characterized by increased exploration and a drive for independence. This paper investigates how confirmatory reinforcement learning (RL) evolves from adolescence to early adulthood, specifically asking if this bias is attenuated during a developmental stage where exploration is crucial.
Methods
The researchers employed a standard probabilistic instrumental RL task. Participants (aged 11-32) repeatedly chose between pairs of novel options with varying reward/punishment probabilities, aiming to maximize monetary rewards. The task included three conditions:
- Asymmetric: One option consistently offered a higher win probability.
- Reversal: Probabilities were stable for the first half of trials and then reversed.
- Symmetric: Both options had equal win probabilities.
Behavioral analyses focused on accuracy, preferred choice rate, win-stay/lose-shift asymmetry, decision times, and outcome observation times. Computational modeling used a confirmation model, which allows for separate learning rates for confirmatory and disconfirmatory evidence, to capture age-related differences.
Datasets
The study used data from 77 participants (aged 11-32 years, 4 men, mean age = 16.26). The data from adult participants (N=20) had been previously published. No new benchmarks were introduced.
Key Insights and Ideas
- Adolescence is a transition period for learning: The study highlights that adolescence isn't a static learning phase but a dynamic period where learning strategies shift towards greater confirmation.
- Confirmatory learning is adaptive in stable environments: The findings suggest that learning to rely more on confirming evidence is beneficial when the environment's rules (reward probabilities) are stable.
- Exploration remains important in volatile environments: The age-related advantage in learning is diminished when environmental rules change, indicating that flexibility and disconfirming evidence are still crucial.
- Win-stay/lose-shift asymmetry grows with age: Older participants were more likely to repeat a winning choice and less likely to switch after a losing choice, suggesting a stronger tendency to stick with what has worked.
- Computational models reveal the "why": The confirmation model provided a better fit to the data, showing that age-related differences in learning are best explained by changes in "inverse temperature" (a measure of noise/exploration) rather than the magnitude of the confirmation bias itself.
Key Results
- Accuracy: Accuracy increased linearly with age in stable (asymmetric) learning environments. This age-related benefit was reduced in volatile (reversal) conditions.
- Choice Conservatism: There was a non-significant trend for increased conservatism (sticking with a preferred option) with age in symmetric trials, but this did not reach statistical significance.
- Win-Stay/Lose-Shift: A significant increase in win-stay/lose-shift asymmetry was observed with age. Participants were more likely to stay after a win and this tendency increased with age, while the tendency to switch after a loss did not change significantly with age.
- Decision and Outcome Observation Times: Decision times showed no overall age effect, but increased with age in symmetric trials. Outcome observation times decreased linearly with age.
- Computational Model: The confirmation model best explained the data, and its fit improved with age. Age-related differences were best captured by decreases in inverse temperature (noise/exploration), not by changes in the confirmation bias magnitude.
Limitations and Future Work
- Sample Size: The study acknowledges that it might have been underpowered, particularly for detecting smaller age-related effects in older participants or for investigating choice conservatism in symmetric trials.
- Gender Bias: The sample was predominantly female, limiting the generalizability of findings to males. Future research should investigate potential gender differences in RL.
- Cross-sectional Design: The study used a cross-sectional design, which conflates age-related effects with inter-individual differences. Longitudinal studies are recommended to track developmental changes more directly.
- Future Directions: The authors suggest that better-powered studies and longitudinal approaches are needed. Further investigation into how environmental factors (like option differentiability) interact with age-related learning changes is also warranted.
Summary updated