Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning.

Amy Walsh, Peter Dayan, Andreas Olsson, Marc Guitart‐MasipPsyArXiv 2022 · DOI

Abstract

Unavoidable stress elicits perceived lack of controllability and learned helplessness, which is a risk-factor for depression. Stress, anxiety, and depression can all alter reward learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning of optimal choices. Uncontrollable stress also predicted a greater proportion of probabilistic errors following negative feedback for correct choices, which reflected interpretation of ambiguous negative feedback as a change in task reward contingencies. Moreover, this effect of uncontrollable stress on probabilistic errors was mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression.

Introduction

While it is well-established that stress and anxiety can disrupt reward-based learning, the underlying cognitive mechanisms have remained unclear. Does stress impair motivation, reward processing, or the internal models we build of our environment? This study investigates "uncontrollable stress"—the perception that one's actions do not influence outcomes—and its impact on expectations of "environmental volatility" (the frequency of rule changes in the world). This research was conducted during the first wave of the COVID-19 pandemic, providing a unique context of widespread, uncontrollable uncertainty.

Methods

The study involved 427 participants recruited via Prolific in April 2020. Participants completed psychological assessments, including the Perceived Stress Scale (PSS), State-Trait Anxiety Inventory (STAI), and PHQ-9.

Behavioral Task

Participants performed a "three-armed-bandit" probabilistic reward learning task. In each trial, they chose between three images to maximize rewards. The task included two conditions:

  1. Signalled Task: Changes in the optimal choice were explicitly indicated.
  2. Reversal Task: Changes in the optimal choice occurred silently, requiring participants to infer shifts from feedback.
Trial structure for the three-armed-bandit probabilistic reward learning task.

Computational Modeling

The researchers compared standard Reinforcement Learning (RL) models with Hidden Markov Models (HMMs). The HMMs treated the "best" choice as a hidden state that could transition over time. A key parameter, the transition parameter ($tr$), was used to quantify participants' internal beliefs about environmental volatility.

Key Results and Insights

The study found that uncontrollable stress and state anxiety significantly impaired the ability to learn optimal choices, specifically in the reversal task where rules were ambiguous.

Anxiety as a Volatility Filter

A primary finding was that anxious individuals interpreted ambiguous negative feedback (e.g., no reward for a correct choice) as a sign that the environment had changed, rather than as random noise. This led to "probabilistic errors," where participants switched away from correct choices prematurely.

Statistical Findings

Uncontrollable stress and state anxiety were significant predictors of learning deficits. As shown in the results table, the interaction between these psychological states and trial progression was statistically significant.

Statistical models showing the effects of uncontrollable stress and state anxiety on task performance.

The Mediation Pathway

Mediation analysis revealed a specific psychological chain: Uncontrollable Stress → State Anxiety → Increased Transition Parameter ($tr$) → Learning Deficits. The $tr$ parameter from the HMM mediated 45% of the relationship between state anxiety and performance impairments. Notably, these effects were absent in the signalled task, suggesting that stress specifically impacts the interpretation of environmental uncertainty.

Discussion

The findings suggest that humans approach complex environments by inferring hidden states rather than simple reward histories. Under conditions of uncontrollable stress, anxiety warps this inference process, leading to an "overly volatile" representation of the world.

Limitations and Future Directions

While the study provides strong evidence for this cognitive pathway, it is correlational. Future experimental work could induce uncontrollability to confirm causal links. Additionally, the low test-retest reliability of the behavioral task suggests that while the observed effects are significant, the true impact of these psychological traits on cognition might be even more pronounced. Understanding these biases is crucial, as they may represent a precursor to anhedonia and clinical depression.

Summary updated

Abstract

Unavoidable stress elicits perceived lack of controllability and learned helplessness, which is a risk-factor for depression. Stress, anxiety, and depression can all alter reward learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning of optimal choices. Uncontrollable stress also predicted a greater proportion of probabilistic errors following negative feedback for correct choices, which reflected interpretation of ambiguous negative feedback as a change in task reward contingencies. Moreover, this effect of uncontrollable stress on probabilistic errors was mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression.
BibTeX
@misc{walsh2022anxiety,
  author = {Amy Walsh and Peter Dayan and Andreas Olsson and Marc Guitart‐Masip},
  title = {{Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning.}},
  year = {2022},
  doi = {10.31234/osf.io/jbuqt},
  howpublished = {PsyArXiv},
}
Figure