TL;DR
This study challenges the prevailing idea that we allocate cognitive effort based on the average reward rate of our environment. Instead, it reveals that recent performance history, specifically accuracy, is a much stronger predictor of how much effort we exert. This suggests our brains might be prioritizing task mastery over environmental opportunities when deciding how hard to think.
What Problem Does This Paper Address?
Humans, like with physical or financial resources, tend to avoid expending cognitive effort. This raises a fundamental question: how do we decide when to exert this effort? Current theories, often rooted in reinforcement learning, propose that we weigh the costs of cognitive effort against potential rewards. A prominent idea is that we use the average reward rate of our environment as a proxy for "opportunity cost" – the idea that spending time on one task means missing out on others. This suggests that in low-reward environments, we should exert more effort to maximize our chances of getting a reward, as opportunities are scarce. While previous studies supported this, this paper aims to replicate and refine these findings, questioning whether the average reward rate is truly the primary driver of cognitive effort allocation.
Methods
The researchers employed a modified Simon task, a well-established cognitive control task. Participants were presented with a stimulus and had to respond based on its color, with the location of the stimulus sometimes misleading (incongruent trials), requiring more cognitive effort. To manipulate the perceived reward rate, they used pseudo-random sequences of potential rewards for correct responses.
The core of their analysis involved mixed-effects linear and logistic regressions. They modeled accuracy and reaction times (RT) as dependent variables, with factors including reward rate, trial congruency, previous trial performance (error, congruency), response repetition, and trial number. A key aspect was comparing the predictive power of the average reward rate against recent performance history (specifically, an "accuracy rate" calculated using a similar update rule as reward rate, but based on binary reward outcomes).
Datasets
The study used data collected from 68 healthy volunteers performing the Simon task. No new datasets or benchmarks were introduced.
Key Insights and Ideas
- Performance History Trumps Reward Rate: The most striking finding is that recent performance accuracy (how often participants got it right) was a significantly better predictor of cognitive effort than the average reward rate of the environment. This suggests our internal "performance monitor" might be more influential than external reward signals.
- Insensitivity to Immediate Stakes: Participants were largely insensitive to the reward magnitude on the current trial. This implies that while they might track rewards in general, the immediate potential gain didn't directly dictate their effort level in that specific moment.
- Accuracy Rate as a Performance Signal: The "accuracy rate" metric, which essentially tracks the recent history of correct responses, proved more powerful than the reward rate. This suggests that participants might be using their own performance as a signal to adjust effort, perhaps to maintain a certain level of accuracy or avoid errors.
- Reinterpreting Previous Findings: The study calls for a reinterpretation of prior research that linked average reward rate to cognitive effort. It suggests that these effects might have been driven by the underlying performance history, which is often correlated with reward rate.
Key Results
- Replication of Reward Rate Effect (with a Twist): The study replicated the finding that participants exerted more cognitive effort (indicated by higher accuracy) when the average reward rate was low, and less effort (lower accuracy) when it was high. However, this effect was significantly weaker and less predictive than the effect of accuracy rate.
- Accuracy Rate Predicts Performance: A higher accuracy rate was associated with lower accuracy on incongruent trials, indicating a downward shift in the speed-accuracy trade-off. This suggests that when participants are performing well, they might be more prone to errors, possibly due to reduced effort.
- No Effect of Immediate Reward Stake: The amount of reward available on a given trial did not significantly affect accuracy or reaction times.
- Model Evidence Favors Accuracy: When comparing models, the one incorporating accuracy rate as a predictor significantly outperformed the model using reward rate, both for predicting accuracy and reaction times.
Limitations and Future Work
- Environmental Richness Heuristic: While the study found accuracy rate to be a better predictor, the authors acknowledge that environmental richness (as proxied by reward rate) might still serve as a heuristic for assessing opportunity costs. Future work should explore this relationship more carefully, ensuring effective reward rate manipulations.
- Dissociating Performance and Reward: The study suggests that participants might be tracking performance (errors) rather than rewards themselves. Future research could further dissociate these factors, perhaps by manipulating reward feedback independently of performance feedback.
- Fatigue: The authors briefly consider fatigue as a potential explanation for performance changes but note that performance generally improved, suggesting linear fatigue effects are unlikely to be the primary driver. However, more complex fatigue models could be explored.
- Mechanism of Accuracy Rate Influence: The precise mechanism by which accuracy rate influences cognitive effort remains to be fully elucidated. Is it a direct performance-monitoring signal, or something else? Further investigation is needed.
Summary updated