TL;DR
This paper investigates how our brains arbitrate between automatic, stimulus-driven (Pavlovian) and goal-directed, action-contingent (instrumental) behaviors, especially when they conflict. Using computational modeling and neuroimaging (EEG and fMRI), they found neural signals in frontal cortex that track our inferred sense of control over outcomes, which in turn predicts the strength of Pavlovian influences on our actions.
What Problem Does This Paper Address?
We often act based on learned associations between stimuli and outcomes (Pavlovian learning) or based on the consequences of our specific actions (instrumental learning). While both are crucial for survival, they can sometimes clash. For instance, a cue that previously predicted reward might make us approach it, even if our goal-directed behavior dictates we should avoid it to prevent a punishment. This phenomenon, known as "Pavlovian misbehavior," highlights the need to understand how the brain arbitrates between these two systems. Previous work has suggested that this arbitration is influenced by our perception of control over outcomes – if we feel we can influence the outcome, instrumental control might dominate; if we feel powerless, automatic Pavlovian responses might take over. This paper aims to find the neural signatures of this arbitration process, specifically how the brain tracks inferred controllability and uses this information to modulate the balance between Pavlovian and instrumental action selection.
Methods
The researchers employed a multi-pronged approach:
- Computational Modeling: They used a Bayesian framework to model how individuals learn and make decisions in a Go/NoGo task. This model posits that an "arbitrator" dynamically weighs Pavlovian and instrumental action values based on inferred controllability. They compared a "Fixed Bayesian" model (where this weight is static) with an "Adaptive Bayesian" model (where the weight updates based on experience).
- Behavioral Analysis: They analyzed data from two existing studies (one EEG, one fMRI) to see if the Adaptive Bayesian model best explained participants' behavior, particularly their "go bias" (tendency to choose the "Go" action).
- Neuroimaging Analysis (EEG): They examined midfrontal theta power (a brainwave oscillation in the 4-8 Hz range) in relation to the dynamically estimated Pavlovian weight from the computational model.
- Neuroimaging Analysis (fMRI): They investigated activity in two specific frontal brain regions: the inferior frontal gyrus (IFG) and the ventromedial prefrontal cortex (vmPFC), again correlating this activity with the estimated Pavlovian weight.
Datasets
The study re-analyzed data from two previously published studies:
- An EEG study by Cavanagh et al. (2013) involving 34 participants.
- An fMRI study by Guitart-Masip et al. (2012) involving 30 participants.
The experimental design in both studies was a Go/NoGo task where participants learned to associate visual cues with rewards or punishments, and to either press a button (Go) or withhold a button press (NoGo) to achieve the best outcome. The key manipulation involved varying the predictability of outcomes based on actions, allowing for the inference of controllability.
Key Insights and Ideas
- Controllability as the Arbitrator: The core idea is that our subjective sense of control over outcomes is a critical factor in determining whether we rely more on automatic Pavlovian responses or deliberate instrumental actions.
- Dynamic Arbitration: The brain doesn't just have a fixed bias; it dynamically adjusts the balance between Pavlovian and instrumental influences on a trial-by-trial basis, adapting to inferred changes in controllability.
- Neural Correlates of Inferred Controllability: Specific brain signals appear to track this inferred controllability, providing a neural basis for the computational model's predictions.
- Frontal Cortex Involvement: The frontal cortex plays a crucial role in this arbitration process, with different sub-regions potentially handling distinct aspects of the computation.
Key Results
- Behavioral Model Fit: The Adaptive Bayesian model, which allows the Pavlovian influence to change dynamically with inferred controllability, provided a significantly better fit to the behavioral data from both EEG and fMRI studies compared to simpler models.
- EEG: Midfrontal Theta Power: Midfrontal theta power was found to decrease as the Pavlovian weight (reflecting inferred uncontrollability) increased. This suggests that higher theta power is associated with a greater sense of control and a weaker reliance on Pavlovian biases.
- fMRI: Inferior Frontal Gyrus (IFG): The IFG showed a stronger response to "NoGo" actions compared to "Go" actions, but this effect was most pronounced when inferred controllability was low (i.e., when the Pavlovian weight was high). This aligns with the IFG's known role in inhibiting prepotent responses, which is more critical when automatic biases are strong.
- fMRI: Ventromedial Prefrontal Cortex (vmPFC): The vmPFC activity scaled with the Pavlovian weight, showing a stronger response when Pavlovian influences were high. It also responded more to rewards than punishments, suggesting it integrates both valence and the degree of Pavlovian influence.
Limitations and Future Work
The authors acknowledge several limitations and suggest future research directions:
- Midcingulate Cortex: The fMRI acquisition in this study did not cover the midcingulate cortex, which is hypothesized to be a key generator of midfrontal theta and involved in "need for control." Future studies using whole-brain acquisition could investigate this directly.
- Outcome-Related Activity: The current analysis focused on cue-evoked responses. Future work could examine neural activity related to outcome receipt to gain a more complete picture of the arbitration process.
- Neural Update Signal: Identifying the specific neural signal that drives the dynamic updates in inferred controllability and Pavlovian bias remains an open question.
- Pavlovian-Instrumental Transfer: The relationship between the Pavlovian biases studied here and the phenomenon of Pavlovian-instrumental transfer needs further empirical and theoretical investigation.
- Theta Power Components: The study focused on total theta power. Future research could explore distinctions between phase-locked and non-phase-locked theta activity, which may carry different computational information.
Summary updated