Dissecting the contribution of recent reward versus recent performance history on cognitive effort allocation

Floortje S. Spronkers, Renée S. Koolschijn, Nathaniel D. Daw, A. Ross Otto, Hanneke E.m. den OudenbioRxiv 2025 · DOI

TL;DR

This study challenges the prevailing idea that we allocate cognitive effort based on the average reward rate of our environment. Instead, it reveals that recent performance history, specifically accuracy, is a much stronger predictor of how much effort we exert. This suggests our brains might be prioritizing task mastery over environmental opportunities when deciding how hard to think.

What Problem Does This Paper Address?

Humans, like with physical or financial resources, tend to avoid expending cognitive effort. This raises a fundamental question: how do we decide when to exert this effort? Current theories, often rooted in reinforcement learning, propose that we weigh the costs of cognitive effort against potential rewards. A prominent idea is that we use the average reward rate of our environment as a proxy for "opportunity cost" – the idea that spending time on one task means missing out on others. This suggests that in low-reward environments, we should exert more effort to maximize our chances of getting a reward, as opportunities are scarce. While previous studies supported this, this paper aims to replicate and refine these findings, questioning whether the average reward rate is truly the primary driver of cognitive effort allocation.

Methods

The researchers employed a modified Simon task, a well-established cognitive control task. Participants were presented with a stimulus and had to respond based on its color, with the location of the stimulus sometimes misleading (incongruent trials), requiring more cognitive effort. To manipulate the perceived reward rate, they used pseudo-random sequences of potential rewards for correct responses.

The core of their analysis involved mixed-effects linear and logistic regressions. They modeled accuracy and reaction times (RT) as dependent variables, with factors including reward rate, trial congruency, previous trial performance (error, congruency), response repetition, and trial number. A key aspect was comparing the predictive power of the average reward rate against recent performance history (specifically, an "accuracy rate" calculated using a similar update rule as reward rate, but based on binary reward outcomes).

Datasets

The study used data collected from 68 healthy volunteers performing the Simon task. No new datasets or benchmarks were introduced.

Key Insights and Ideas

  • Performance History Trumps Reward Rate: The most striking finding is that recent performance accuracy (how often participants got it right) was a significantly better predictor of cognitive effort than the average reward rate of the environment. This suggests our internal "performance monitor" might be more influential than external reward signals.
  • Insensitivity to Immediate Stakes: Participants were largely insensitive to the reward magnitude on the current trial. This implies that while they might track rewards in general, the immediate potential gain didn't directly dictate their effort level in that specific moment.
  • Accuracy Rate as a Performance Signal: The "accuracy rate" metric, which essentially tracks the recent history of correct responses, proved more powerful than the reward rate. This suggests that participants might be using their own performance as a signal to adjust effort, perhaps to maintain a certain level of accuracy or avoid errors.
  • Reinterpreting Previous Findings: The study calls for a reinterpretation of prior research that linked average reward rate to cognitive effort. It suggests that these effects might have been driven by the underlying performance history, which is often correlated with reward rate.

Key Results

  • Replication of Reward Rate Effect (with a Twist): The study replicated the finding that participants exerted more cognitive effort (indicated by higher accuracy) when the average reward rate was low, and less effort (lower accuracy) when it was high. However, this effect was significantly weaker and less predictive than the effect of accuracy rate.
  • Accuracy Rate Predicts Performance: A higher accuracy rate was associated with lower accuracy on incongruent trials, indicating a downward shift in the speed-accuracy trade-off. This suggests that when participants are performing well, they might be more prone to errors, possibly due to reduced effort.
  • No Effect of Immediate Reward Stake: The amount of reward available on a given trial did not significantly affect accuracy or reaction times.
  • Model Evidence Favors Accuracy: When comparing models, the one incorporating accuracy rate as a predictor significantly outperformed the model using reward rate, both for predicting accuracy and reaction times.

Limitations and Future Work

  • Environmental Richness Heuristic: While the study found accuracy rate to be a better predictor, the authors acknowledge that environmental richness (as proxied by reward rate) might still serve as a heuristic for assessing opportunity costs. Future work should explore this relationship more carefully, ensuring effective reward rate manipulations.
  • Dissociating Performance and Reward: The study suggests that participants might be tracking performance (errors) rather than rewards themselves. Future research could further dissociate these factors, perhaps by manipulating reward feedback independently of performance feedback.
  • Fatigue: The authors briefly consider fatigue as a potential explanation for performance changes but note that performance generally improved, suggesting linear fatigue effects are unlikely to be the primary driver. However, more complex fatigue models could be explored.
  • Mechanism of Accuracy Rate Influence: The precise mechanism by which accuracy rate influences cognitive effort remains to be fully elucidated. Is it a direct performance-monitoring signal, or something else? Further investigation is needed.

Summary updated

Abstract

Abstract An extensive body of literature has shown that humans tend to avoid expending cognitive effort, just like for physical effort or financial resources. How then, do we decide whether to put this effort in? Decision-making not only involves choosing our actions, but also the meta-decision of how much cognitive effort to invest in making this choice, weighing the costs of cognitive effort against potential rewards. Popular recent theories, grounded in the field of reinforcement learning, suggest that this cost-benefit trade-off can be informed by the opportunity costs of effort investment, which the brain may approximate by the estimated average reward rate per unit time. It follows from intuition that in a low reward environment, investing cognitive resources in the task at hand will less likely lead to missed opportunities. Recent studies provided support for this idea, showing that people exert more cognitive effort when reward rate is low. Here, we replicate one of the key previous findings but provide an important nuance to this result. Cognitive effort allocation was better explained by participants’ recent performance history (i.e. accuracy rate) than average reward rate. In combination with the observation that participants were insensitive to the reward currently at stake, this invites a reinterpretation of these previous findings and suggests the need for further studies to assess whether environmental richness may indeed serve as a heuristic to modulate cognitive effort allocation.
BibTeX
@misc{spronkers2025dissecting,
  author = {Floortje S. Spronkers and Renée S. Koolschijn and Nathaniel D. Daw and A. Ross Otto and Hanneke E.M. den Ouden},
  title = {{Dissecting the contribution of recent reward versus recent performance history on cognitive effort allocation}},
  year = {2025},
  doi = {10.1101/2025.07.08.663681},
  howpublished = {bioRxiv},
}
Figure