TL;DR
This paper investigates how the brain processes prediction errors (PEs) related to both overall goals and intermediate subgoals. Using fMRI, they found that the medial prefrontal cortex (mPFC) signals these PEs in an "unsigned" manner, meaning it doesn't distinguish between positive and negative errors. Crucially, the mPFC's focus shifts between goal- and subgoal-related PEs depending on task demands, suggesting a flexible attentional mechanism.
What Problem Does This Paper Address?
Human behavior is often hierarchical, involving a main goal broken down into smaller subgoals. Learning and adaptation in such complex tasks rely on prediction error signals – the brain's way of signaling when an outcome deviates from expectation. Previous research had shown that brain regions like the ventral striatum (VS) and medial prefrontal cortex (mPFC) are involved in signaling PEs. However, it was unclear how the brain distinguishes between PEs related to the overarching goal versus those related to intermediate subgoals, especially when both are present simultaneously. This paper aims to disentangle these signals and understand how the mPFC processes this hierarchical information.
Methods
The researchers employed functional magnetic resonance imaging (fMRI) to measure brain activity. They designed two experiments using a hierarchical spatial navigation task where participants controlled a truck to pick up an envelope and deliver it to a house. The key manipulation involved "jumps" of the envelope to new locations, which could create prediction errors at either the subgoal (envelope pickup) or the goal (house delivery) level, or both.
- Experiment I: Focused on eliciting positive subgoal-related PEs.
- Experiment II: Manipulated jumps to elicit both goal- and subgoal-related PEs simultaneously, and also introduced monetary rewards to compare with more standard reward PEs.
They used whole-brain and region-of-interest (ROI) analyses to identify brain areas involved. Standard fMRI preprocessing and analysis pipelines were used, including motion correction, spatial smoothing, and general linear model (GLM) analysis.
Datasets
Not discussed in the paper. The study involved human participants performing a custom-designed task.
Key Insights and Ideas
- mPFC signals PEs in an unsigned manner: The mPFC doesn't differentiate between positive (better than expected) and negative (worse than expected) prediction errors for subgoals. This aligns with previous findings on reward PEs.
- Flexible attention to hierarchy: In situations where both goal and subgoal PEs are present, the mPFC appears to prioritize processing goal-related PEs, suggesting that attention can strongly influence which level of the hierarchy is encoded.
- Dissociation between mPFC and VS: The study observed differences in how the mPFC and VS process these hierarchical PEs, hinting at distinct roles for these regions in reinforcement learning.
- Task context matters: The findings suggest that the mPFC selectively attends to information at different hierarchical levels depending on the specific task context and demands.
Key Results
- Experiment I: Replicated previous findings that the mPFC signals subgoal-related PEs independently of goal PEs. BOLD responses to PEs in mPFC were found to be unsigned.
- Experiment II: When both goal and subgoal PEs were present, the mPFC showed activity primarily related to goal-related PEs, suggesting attentional modulation.
- Behavioral Data: Participants were slower to react to envelope jumps compared to no-jump conditions. Movement error was also higher following jumps.
- Neural Correlates: Increases in BOLD activity were observed in the medial prefrontal cortex (mPFC) and right anterior insula for positive subgoal-related prediction errors. In Experiment II, unsigned goal-related PEs were associated with activity in the left mPFC. Positive monetary reward prediction errors (+25) led to increased BOLD response in the left putamen.
Limitations and Future Work
- Attentional Factors: The authors acknowledge that the observed shift in mPFC activity between goal and subgoal processing in Experiment II might be due to attentional factors. Future work could more directly manipulate attention to confirm this.
- Complexity of Simultaneous PEs: The difficulty in observing subgoal-related PEs in Experiment II when both goal and subgoal PEs were induced simultaneously might stem from the cognitive load of processing two prediction errors from a single source. Future research could explore how the brain handles multiple, simultaneous prediction errors.
- Ecological Validity: While the task is designed to mimic hierarchical decision-making, future studies could explore these mechanisms in more naturalistic, ecologically valid settings.
- Role of VS: The study observed a dissociation between mPFC and VS. Further research could delve deeper into the specific roles of the VS in processing hierarchical prediction errors.
Summary updated