Abstract
Investigating the bases of inter-individual differences in risk-taking is necessary to refine our cognitive and neural models of decision-making and to ultimately counter risky behaviors in real-life policy settings. However, recent evidence suggests that behavioral tasks fare poorly compared to standard questionnaires to measure individual differences in risk-taking. Crucially, using model-based measures of risk taking does not seem to improve reliability. Here, we put forward two possible - not mutually exclusive - explanations for these results and suggest future avenues of research to improve the assessment of inter-individual differences in risk-taking by combining repeated online testing and mechanistic computational models.
The Reliability Paradox in Risk Assessment
In fields like behavioral economics and neuroeconomics, researchers rely on laboratory tasks—such as choosing between a guaranteed $5 or a coin flip for $10—to measure "behavioral phenotypes" of risk-taking. The goal is to use these measures to predict real-world outcomes, like financial decisions or health behaviors.
However, recent large-scale studies have revealed a "reliability paradox": these lab tasks often have low external validity and low test-retest reliability. People's scores change significantly over time, and even applying sophisticated mathematical frameworks like Cumulative Prospect Theory (CPT) fails to fix the inconsistency. Surprisingly, simple personality questionnaires often outperform these behavioral tasks in predicting real-life behavior.
The Multi-Layer Model of Risk-Taking
The authors propose that the failure of behavioral tasks stems from a fundamental misunderstanding of the risk construct. Rather than being a static trait, risk-taking is a dynamic state influenced by multiple layers of temporal variance.
In this multi-layer model, a person’s "momentary risk attitude" at any given time ($t$) is the weighted sum of several factors:
- Trait: Stable genetic and biological baseline.
- Ageing: Slow, long-term shifts.
- State: Medium-term factors like mood or hormonal cycles.
- Circadian: Fast daily fluctuations.
- Random Error: Unpredictable noise.
As illustrated in the figure, this framework explains why a single lab session might fail to capture a person's true underlying trait. Two individuals with different baseline traits might produce identical behavior in a single experimental session (e.g., ES1 or ES2) because transient state and circadian influences happen to overlap at that moment, masking their true underlying differences.
Why Questionnaires Outperform Tasks
The "Questionnaire Advantage" arises because when individuals describe themselves "as they generally are," they perform a mental "temporal averaging" of their past experiences. In contrast, a lab task captures only a single, noisy data point.
Furthermore, many cognitive tasks are designed to minimize individual differences to find robust average effects. This makes them mathematically ill-suited for correlational psychology, which requires capturing variance. Current descriptive models, like CPT, summarize what happened in a single session but lack mechanistic depth—they do not account for how history, feedback, or internal states dynamically alter choices over time.
Synthesis of Evidence
The paper synthesizes findings from several landmark datasets:
- Frey et al. (2017): A large-scale study (~1,500 participants) showing that behavioral tasks correlate less with actual risky behaviors than simple personality scores.
- Enkavi et al. (2018): Demonstrated that reliability issues extend beyond risk-taking to other domains like reinforcement learning.
- Corsetto and Filippin (2013): Provided earlier evidence of low between-task reliability, suggesting different tasks may measure different underlying constructs.
Future Directions: From Static to Dynamic Assessment
To move beyond the current bottleneck of "fixed and passive" task designs, the authors suggest:
- Active and Adaptive Designs: Moving away from static trials toward tasks that adjust parameters online to better "stress test" an individual's specific decision boundaries.
- Repeated Online Testing: Leveraging smartphones to collect longitudinal data. By testing subjects multiple times across different days and times, researchers can use the multi-layer model to statistically separate stable "traits" from transient "states."
- Mechanistic Modeling: Developing models that integrate learning processes and contextual factors rather than assuming risk preferences are static.
Summary updated