TL;DR
This paper extends a computational model of the medial prefrontal cortex (mPFC) to propose that this brain region's primary function is to predict any salient event, not just outcomes of actions. By generalizing the model, the authors show it can explain a broader range of neuroscientific findings, linking mPFC to reinforcement learning and cognitive control.
What Problem Does This Paper Address?
The medial prefrontal cortex (mPFC), particularly the anterior cingulate cortex (ACC), has long been implicated in cognitive control, error detection, and monitoring behavior. A prominent computational model, the "predicted response-outcome" (PRO) model, suggested mPFC learns to predict outcomes contingent on actions. However, recent research has shown mPFC activity even in situations where outcomes are not directly tied to an individual's actions, such as processing task-related cues or novel stimuli. This presented a challenge: either mPFC has separate functions for action-outcome prediction and stimulus processing, or a single, unifying mechanism explains these diverse findings. This paper aims to resolve this by proposing a generalized PRO model that can account for both types of observations.
Methods
The core of this work is a conceptual extension of the existing PRO model. The original PRO model used equations based on temporal difference (TD) learning to predict outcomes based on stimuli and actions. The extended model reinterprets "outcomes" and "stimuli" as "events," defined as any salient sensory input that can be associated with subsequent events. This generalization means the model's equations remain formally identical to the original PRO model, but their interpretation is broadened. The authors then use this generalized model to simulate various experimental paradigms, comparing its output to existing neuroimaging (fMRI, EEG) and single-unit recording data from animal studies. The model's activity is primarily measured as "negative surprise," representing the discrepancy between predicted and observed events.
Datasets
The paper does not introduce new datasets. Instead, it simulates existing experimental paradigms and compares the model's output to published data from various studies. These include:
- Eriksen flanker task data
- Stroop task data
- Mismatch negativity (MMN) paradigms
- Arrow-word Stroop task data
- Stop signal task data
- Monkey single-unit recording data from an expect reward task
Key Insights and Ideas
- Unified Prediction Mechanism: The most significant insight is that a single predictive mechanism, generalized to encompass all salient events (not just action outcomes), can explain a wide array of mPFC functions, from cognitive control to stimulus processing.
- "Event" as the Fundamental Unit: By treating stimuli and outcomes as interchangeable "events" that can be predicted and serve as predictors, the model bridges the gap between action-outcome learning and stimulus-driven prediction.
- mPFC as a General Predictor: This suggests mPFC's core role is not just about what to do, but what is likely to happen next, regardless of whether the individual directly causes it.
- Explaining Diverse Phenomena: The generalized model successfully accounts for phenomena like mismatch negativity (MMN), Bayesian surprise, and item-specific control effects, which were not explicitly the focus of the original PRO model.
Key Results
The paper presents simulation results demonstrating how the generalized PRO model can reproduce patterns of activity observed in various neuroscientific studies:
- Trial Frequency: Model activity reflects the frequency of trial types (congruent vs. incongruent), explaining why conflict-related effects can vary with trial statistics.
- Item-Level vs. Global Control: The model captures how control can depend on item-specific frequencies of incongruent trials, not just global task statistics.
- Mismatch Negativity (MMN): The model explains mPFC MMN as the surprising absence of a predicted stimulus, consistent with its "negative surprise" signal.
- Informative vs. Uninformative Cues: The model predicts increased activity for informative cues (predicting specific trial types) and reduced activity for task-related events when preceded by such cues.
- Bayesian Surprise: The model's activity aligns with Bayesian surprise signals in the ACC during stop-signal tasks, reflecting changes in the estimated likelihood of events.
- Single-Unit Activity: The model's predictions for monkey ACC neurons match observed responses to unexpected reward cue switches, interpreted as deviations from expected events.
Limitations and Future Work
- Temporal Representation: A key limitation acknowledged is the model's reliance on consistent inter-event timing for convergence, which is a simplification of how the brain processes time. Future work should explore more realistic temporal representations.
- Model-Based RL Integration: While the model is consistent with model-based reinforcement learning (RL), it is not itself a full model-based RL algorithm. The authors suggest future work should investigate how mPFC's prediction error signal might interact with other brain regions (like dorsolateral PFC or the hippocampus) involved in learning and maintaining world models.
- Distinction from OFC: The generalized PRO model blurs the previously appealing distinction between mPFC (action values) and orbitofrontal cortex (OFC) (stimulus values). Future research could explore how OFC's state representations might inform mPFC's predictions, and how mPFC's prediction errors might inform OFC's state tracking.
- Attention Allocation: The authors propose that mPFC's prediction error signal might play a role in allocating attention to relevant stimuli, a hypothesis that warrants further investigation.
Summary updated