Sympathetic involvement in time-constrained sequential foraging

Neil M. Dundon, Neil Garrett, Viktoriya Babenko, Matt Cieslak, Nathaniel D. Daw, Scott T. GraftonCognitive 2020 · DOI

TL;DR

This study reveals that the sympathetic nervous system plays a crucial, adaptive role in learning about deteriorating environments during sequential decision-making tasks. Specifically, heightened sympathetic activity is linked to faster learning from negative feedback and improved performance in challenging conditions, suggesting a sophisticated interplay between our physiological stress response and cognitive adaptation.

What Problem Does This Paper Address?

Navigating environments with uncertain future opportunities and time costs is a fundamental challenge. Think of a contract worker deciding which project to take: a high-paying one might mean missing out on even better opportunities later, while a low-paying one might be the best available if the market is bad. This is the essence of sequential foraging, where decisions involve a trade-off between immediate rewards and the opportunity cost of time. Previous research has shown that humans tend to learn about the "richness" (overall reward rate) of such environments with a bias towards positive information – we're more optimistic about improvements than we are cautious about declines. However, how the body's stress response, particularly the sympathetic nervous system, influences this learning process, especially in dynamic or deteriorating environments, has been unclear. Existing studies often use slower measures of stress (like cortisol) that don't capture the rapid, trial-by-trial adjustments needed for foraging decisions. This paper aims to bridge this gap by investigating the real-time role of sympathetic activation in learning and decision-making within a time-constrained foraging task.

Methods

The researchers employed a sophisticated approach combining behavioral tasks with continuous physiological monitoring.

  • Prey Selection Task: Participants played a computerized video game simulating a sequential foraging scenario. They had to decide whether to "capture" approaching "invaders" that offered varying rewards (fuel) and incurred different time costs. The game was played in two distinct environments: a "boom" environment with a high concentration of profitable invaders and a "downturn" environment with more low-profit invaders. This setup allowed for the manipulation of environmental richness and the observation of learning dynamics.
  • Cardiac Physiology Measurement: Continuous electrocardiogram (ECG) and impedance cardiogram (ICG) data were recorded. From these signals, the researchers extracted two key physiological measures:
  • Pre-ejection Period (PEP): This is a measure of myocardial contractility, which is strongly influenced by sympathetic nervous system activity. Shorter PEP indicates increased sympathetic drive.
  • Heart Rate (HR): This measure, influenced by both sympathetic and parasympathetic systems, was used to control for potential confounds.
  • Data Analysis:
    • Behavioral Analysis: The study first replicated previous findings of an "asymmetric learning bias" where participants updated their beliefs about environmental richness more readily with positive than negative information.
  • Physiological-Behavioral Correlation: Trial-by-trial analyses examined how physiological states (PEP and HR) correlated with choices, environmental richness, and their derivatives.
  • Computational Modeling: Reinforcement learning models were used to quantify learning parameters, specifically investigating how sympathetic activation modulated the learning rate for positive and negative prediction errors.
  • Optimal Performance: The study assessed whether physiological states predicted optimal performance, defined as the ability to adapt capture strategies between the boom and downturn environments.

Datasets

The study did not introduce new benchmark datasets. Instead, it utilized data collected from 20 participants performing the custom-designed "Prey Selection Task." This task was based on a paradigm previously used in related research (e.g., Garrett & Daw, 2019), allowing for direct comparison and replication of findings.

Key Insights and Ideas

  • Sympathetic System as a "Deterioration Detector": The most striking insight is that sympathetic activation (measured by shorter PEP) specifically tracks the deterioration of the environment (i.e., a decrease in reward rate). It doesn't seem to respond as strongly to environmental improvements. This suggests a specialized role for the sympathetic system in signaling negative changes.
  • Faster Learning from Bad News: This sympathetic "deterioration detection" translates into adaptive learning. When sympathetic drive is high, participants learn more quickly from negative prediction errors (i.e., when they encounter less rewarding outcomes than expected). This means they adapt their behavior faster when the environment gets worse.
  • Predicting Optimal Performance: Crucially, this enhanced learning from negative feedback, driven by sympathetic engagement, was predictive of optimal performance. Participants with higher sympathetic drive during the early, critical learning phases of the downturn environment were better at adjusting their capture strategy to maximize rewards.
  • Asymmetric Learning Confirmed and Extended: The study confirmed the previously observed "naïve perseverance of optimism" (a bias towards positive information) and extended it by showing how physiological states can modulate this asymmetry, particularly for negative information.

Key Results

  • Environmental Deterioration and PEP: A significant negative correlation was found between the rate of change in environmental richness and PEP (dPEP/dt). This means as the environment's reward rate decreased, sympathetic activity (shorter PEP) increased. This effect was specific to deterioration, not improvement.
  • Sympathetic Modulation of Negative Learning: The "Asymmetry Phys a¯" model showed that learning from negative prediction errors (a¯) was significantly modulated by PEP. Specifically, higher sympathetic drive (shorter PEP) led to a greater adjustment of beliefs following negative outcomes.
  • PEP and Optimal Strategy Shift: In the first half of the task (early learning), higher sympathetic drive (D-B Drive PEP) was a significant positive predictor of optimal performance (higher D-B Mid score), indicating better adaptation to the downturn environment. This effect was not observed for HR.
  • HR and Value: Heart rate (HR) was found to scale negatively with low-value capture, suggesting it might be involved in suppressing the capture of less rewarding items when the environment is generally poor.

Limitations and Future Work

  • Causality: While the study demonstrates strong associations, establishing definitive causal links between sympathetic activation and adaptive learning requires further investigation. Future research could employ interventions (e.g., pharmacological or neurostimulation) to directly manipulate sympathetic activity and observe its effects on learning and decision-making.
  • Cortical Mechanisms: The precise neural circuits underlying this sympathetic involvement remain to be elucidated. The authors suggest that areas like the anterior cingulate cortex (ACC), known for its role in conflict monitoring and decision-making, might be involved. Future studies could combine physiological measures with neuroimaging techniques (fMRI, EEG) to map these interactions.
  • Generalizability: The study focused on a specific foraging task. Future work should explore whether this adaptive role of the sympathetic system extends to other types of decision-making tasks, particularly those that don't involve incremental belief updating or are less dynamic.
  • Autonomic System Complexity: The study primarily focused on the sympathetic branch. Future research could explore the interplay between sympathetic and parasympathetic systems, as well as other stress response pathways (like the HPA axis), to gain a more comprehensive understanding of autonomic regulation in decision-making. The authors also note that HR is influenced by both branches, making its interpretation complex.

Summary updated

Abstract

Appraising sequential offers relative to an unknown future opportunity and a time cost requires an optimization policy that draws on a learned estimate of an environment's richness. Converging evidence points to a learning asymmetry, whereby estimates of this richness update with a bias toward integrating positive information. We replicate this bias in a sequential foraging (prey selection) task and probe associated activation within the sympathetic branch of the autonomic system, using trial-by-trial measures of simultaneously recorded cardiac autonomic physiology. We reveal a unique adaptive role for the sympathetic branch in learning. It was specifically associated with adaptation to a deteriorating environment: it correlated with both the rate of negative information integration in belief estimates and downward changes in moment-to-moment environmental richness, and was predictive of optimal performance on the task. The findings are consistent with a framework whereby autonomic function supports the learning demands of prey selection.
BibTeX
@inproceedings{dundon2020sympathetic,
  author = {Neil M. Dundon and Neil Garrett and Viktoriya Babenko and Matt Cieslak and Nathaniel D. Daw and Scott T. Grafton},
  title = {{Sympathetic involvement in time-constrained sequential foraging}},
  booktitle = {Cognitive},
  year = {2020},
  doi = {10.3758/s13415-020-00799-0},
}
Figure