TL;DR
This paper proposes a computational model where serotonin's role in behavioral inhibition helps prune decision trees of potential thoughts, preventing exploration of negative outcomes. This model explains how reduced serotonin can lead to overly optimistic valuations and large negative prediction errors, offering a new perspective on depression and anxiety treatments.
What Problem Does This Paper Address?
The paper tackles a long-standing puzzle in neuroscience and psychiatry: the seemingly contradictory roles of serotonin (5-HT). While 5-HT is implicated in predicting aversive events and in behavioral inhibition, it's also linked to negative mood states. Furthermore, the effectiveness of SSRIs (which boost 5-HT) in treating depression, despite 5-HT's association with negative outcomes, presents a paradox. The authors aim to reconcile these observations by modeling how 5-HT-mediated inhibition influences learning and decision-making in the context of affectively charged thoughts.
Methods
The authors developed a highly simplified computational model of "trains of thought," where thoughts are treated as actions leading from one belief state to another, ultimately reaching terminal outcome states with associated affective values (positive or negative). The core of their model is the introduction of a serotonin-dependent inhibitory mechanism: when a train of thought encounters a state with a negative value, serotonin (modeled as a parameter $\alpha_{5HT}$) increases the probability of terminating that thought process. This mechanism effectively prunes branches of the decision tree that lead to negative outcomes.
They used a Markov decision process framework to formalize this. The model learns state values ($V(s)$) through a temporal difference learning rule. They then simulate manipulations of serotonin levels by altering the $\alpha_{5HT}$ parameter, both during learning and after values have been acquired, to observe the effects on learning, prediction errors, and overall affective outcomes. They also explored variations, including recall bias (favoring negative memories) and reward seeking (dopamine-like approach behavior).
Datasets
Not discussed in the paper. The study relies on a simulated environment and computational modeling rather than empirical datasets.
Key Insights and Ideas
- Serotonin as a "Pruner" of Negative Thoughts: The central idea is that serotonin's role in behavioral inhibition acts like a pruning mechanism in a decision tree of thoughts. It cuts off pathways leading to predicted negative outcomes, preventing the organism from dwelling on them.
- Optimistic Valuation Bias: This inhibition, while adaptive, leads to an "optimistic bias" in learned state values. Negative states are under-explored, and their true negative value is underestimated because the organism is less likely to experience them directly.
- The Cost of Reduced Inhibition: When serotonin levels drop (e.g., due to tryptophan depletion), this inhibitory "crutch" is removed. The model predicts that this leads to:
- Under-exploration of negative states: The organism has to actively learn to avoid bad situations rather than relying on reflexive inhibition.
- Mismatched Policy and Value: A discrepancy arises between what the organism thinks it will experience (overly optimistic valuations) and what it actually experiences (more negative outcomes).
- Large Negative Prediction Errors: The unexpected arrival at negative terminal states, which were previously suppressed, results in significant negative prediction errors, contributing to negative mood.
- Explaining SSRI Efficacy: This framework suggests that SSRIs, by boosting serotonin and thus enhancing inhibition, help to re-establish the pruning mechanism. This prevents the organism from dwelling on negative thoughts and experiencing large negative prediction errors, thereby alleviating depressive symptoms.
Key Results
- Behavioral Inhibition and Value Estimation: With high serotonin ($\alpha_{5HT} = 20$), the model shows a bias in value estimation, particularly for negatively valued states, which are poorly estimated due to under-exploration (Figure 3C, 3D). However, average reward during learning is enhanced (Figure 3F).
- Tryptophan Depletion (Reduced Inhibition): Reducing $\alpha_{5HT}$ after learning leads to a significant decrease in average affective return (utility) (Figure 4A). This is accompanied by greatly increased negative prediction errors when transitioning to aversive states (Figure 4C).
- Recall Bias Amplifies Negative Effects: Introducing a bias towards recalling negative states (negative $\beta$) exacerbates the disutility, especially when serotonin is low ($\alpha_{5HT} = 0$), leading to a substantial negative impact on average utility (Figure 5A).
- Reward Seeking Interaction: Dopaminergic-like reward seeking behavior (controlled by parameter $\theta$) can either mitigate or worsen the effects of reduced serotonin, depending on its strength and direction (Figure 5B).
Limitations and Future Work
The authors acknowledge that their model is a highly simplified representation of complex brain processes. They note that:
- Generalization: The model cannot accommodate all the diverse and confusing roles of 5-HT.
- Specific Distinctions: It doesn't speak to distinctions in processing specific facial expressions (e.g., fear vs. disgust).
- Alternative Theories: They mention alternative views of 5-HT's role, such as controlling the discount factor for future rewards.
- BIS Nuances: Their model captures a subset of Gray's Behavioral Inhibition System (BIS) and doesn't fully account for BIS's engagement by conflict or its role in initiating defensive behaviors.
- Dopamine-Serotonin Interaction: Disentangling the complex interplay between dopamine and serotonin remains a pressing task.
Future work suggested includes:
- Experimental Validation: Designing experiments to test predictions about how TrD or SSRIs affect sequential decision-making tasks, particularly in individuals with different serotonin transporter gene polymorphisms (5HTTLPR).
- Modeling Complex Environments: Exploring more complex environments where rewards are hidden behind punishments.
- Modeling Interruption: Modeling the interruption of thought processes as a switch between different Markov decision problems.
- Normative Meta-Adaptation: Investigating whether reduced serotonin could be a normative meta-adaptation to environmental statistics, rather than purely pathological.
- Optimal Inhibition: Formalizing the trade-off between accurate knowledge about state values and the cost of learning, to determine optimal levels of inhibition.
Summary updated