Differential Effects of Psychotic Illness on Directed and Random Exploration

James A. Waltz, Robert C. Wilson, Matthew A. Albrecht, Michael J. Frank, James M. GoldComputational 2020 · DOI

TL;DR

This study reveals that individuals with schizophrenia exhibit deficits in "directed exploration" (information-seeking) but not "random exploration" (behavioral variability) when making decisions under uncertainty. This dissociation suggests that schizophrenia impacts goal-directed learning and decision-making, with a subset of patients showing extreme "ambiguity aversion" that significantly impairs their ability to explore.

What Problem Does This Paper Address?

Schizophrenia is known to be associated with various decision-making deficits, but the precise nature and scope of these impairments remain unclear. One critical aspect of decision-making is the "explore/exploit dilemma": deciding whether to stick with known, rewarding options (exploit) or try out uncertain ones to gain more information (explore). This paper focuses on understanding how individuals with schizophrenia navigate this dilemma, specifically by differentiating between two exploration strategies: directed exploration (actively seeking information to reduce uncertainty) and random exploration (exploring due to general behavioral variability). Previous research has shown that healthy individuals use both strategies, but it's unknown how these are affected in schizophrenia, especially considering the disorder's known impact on cognitive functions crucial for learning and decision-making.

Methods

The researchers employed the Horizon Task, a novel gambling task designed to quantify directed and random exploration. In this task, participants make choices between virtual slot machines with unknown reward probabilities. The "horizon" (number of future trials) is manipulated to alter the value of exploration: a short horizon favors exploitation, while a long horizon favors exploration.

Key methodological aspects include:

  • Model-Free Measures: Directed exploration was quantified by the probability of choosing the more informative option in an "unequal information" condition, while random exploration was measured by the frequency of choosing the lower-mean option in an "equal information" condition.
  • Model-Based Analysis: A modified Kalman filter model was used to estimate underlying learning and decision parameters, including information-seeking (directed exploration) and reward-seeking (random exploration) weights.
  • Subgroup Analysis: Participants were also analyzed based on their level of "ambiguity aversion" (AA), a tendency to avoid uncertain options.

Datasets

The study used data from:

  • 108 patients diagnosed with schizophrenia or schizoaffective disorder (PSZ).
  • 33 healthy volunteers (HVs).

No new benchmarks were introduced; the study utilized an existing task (Horizon Task) and standard cognitive assessments.

Key Insights and Ideas

  • Dissociation in Exploration: The most striking finding is that schizophrenia selectively impairs directed exploration, while random exploration remains intact. This suggests that the disorder affects the strategic, goal-oriented aspect of information seeking, rather than general behavioral variability.
  • Subtypes of Impairment: A significant portion of patients (21%) exhibited "extreme ambiguity aversion," meaning they almost never chose informative options, even when beneficial. This highlights a distinct subgroup with severe exploration deficits.
  • Cognition and Exploration Link: Directed exploration was strongly correlated with measures of intellectual function (IQ, working memory, processing speed). This suggests that cognitive capacity plays a crucial role in the ability to engage in strategic information seeking.
  • Symptoms and Exploration Link: Random exploration, not directed exploration, was linked to positive symptom severity. This implies that while positive symptoms might drive unsystematic variability, they don't necessarily impair the ability to seek information strategically.

Key Results

  • Reduced Directed Exploration: Patients with schizophrenia showed significantly less directed exploration compared to healthy controls. This deficit was evident across different task conditions.
  • Intact Random Exploration: There were no significant differences in random exploration between patients and controls.
  • Ambiguity Aversion Subgroup: 21% of patients displayed extreme ambiguity aversion, showing virtually no directed exploration. This subgroup also had lower cognitive scores.
  • Cognitive Correlates: Higher scores on measures of intellectual function (WASI IQ, MATRICS composite score) were associated with greater directed exploration in patients.
  • Symptom Correlates: Positive symptom severity was positively correlated with random exploration.

Limitations and Future Work

  • Task Specificity: The Horizon Task involved only gains and no losses. It's unclear if the observed extreme ambiguity aversion would extend to situations involving potential losses. Future studies could explore this.
  • Task Understanding: While unlikely, the authors acknowledge that the extreme ambiguity aversion could, in theory, reflect a failure to fully understand the task. Future work could use debriefing methods to ensure task comprehension.
  • Medication Effects: The study did not definitively rule out the influence of antipsychotic medications on exploratory behavior. Future research with antipsychotic-naive patients or controlled clinical trials would be beneficial.
  • Neural Correlates: The neural circuits underlying these dissociable exploration deficits remain unknown. Determining how these behavioral changes relate to specific brain functions is a key direction for future research.

Summary updated

Abstract

Schizophrenia is associated with a number of deficits in decision-making, but the scope, nature, and cause of these deficits are not completely understood. Here we focus on a particular type of decision, known as the <i>explore/exploit</i> dilemma, in which people must choose between exploiting options that yield relatively known rewards and exploring more ambiguous options of uncertain reward probability or magnitude. Previous work has shown that healthy people use two distinct strategies to decide when to explore: directed exploration, which involves choosing options that would reduce uncertainty about the reward values (information seeking), and random exploration (exploring by chance), which describes behavioral variability that is not goal directed. We administered a recently developed gambling task designed to quantify both directed and random exploration to 108 patients with schizophrenia (PSZ) and 33 healthy volunteers (HVs). We found that PSZ patients show reduced directed exploration relative to HVs, but no difference in random exploration. Moreover, patients' directed exploration behavior clusters into two qualitatively different behavioral phenotypes. In the first phenotype, which accounts for the majority of the patients (79%) and is consistent with previously reported behavior, directed exploration is only marginally (but significantly) reduced, suggesting that these patients can use directed exploration, but at a slightly lower level than community controls. In contrast, the second phenotype, comprising 21% of patients, exhibit a form of "extreme ambiguity aversion," in which they almost never choose more informative options, even when they are clearly of higher value. Moreover, in PSZ, deficits in directed exploration were related to measures of intellectual function, whereas random exploration was related to positive symptoms. Taken together, our results suggest that schizophrenia has differential effects on directed and random exploration and that investigating the explore/exploit dilemma in psychosis patients may reveal subgroups of patients with qualitatively different patterns of exploration.
BibTeX
@inproceedings{waltz2020differential,
  author = {James A. Waltz and Robert C. Wilson and Matthew A. Albrecht and Michael J. Frank and James M. Gold},
  title = {{Differential Effects of Psychotic Illness on Directed and Random Exploration}},
  booktitle = {Computational},
  year = {2020},
  doi = {10.1162/cpsy\_a\_00027},
}
Figure