TL;DR
This study reveals that the subjective cost of conflict influences our decisions, sometimes even overriding our learned value representations. Interestingly, the impact of conflict on learning depends on its type: motivational conflicts (internal drives vs. instructions) hinder learning, while conflicts with external distractors do not appear to affect it.
What Problem Does This Paper Address?
We constantly make decisions, weighing potential rewards against the effort or costs involved. While we know that physical and mental effort act as costs, influencing our choices, it's less clear if experiencing conflict itself is perceived as a cost in value-based decision-making. This paper investigates whether the aversive nature of conflict, particularly when it arises from external stimuli conflicting with our internal value assessments, influences our choices and subsequent learning. The authors aim to disentangle the effects of different types of conflict (internal vs. external) on both decision-making and instrumental learning.
Methods
The researchers employed a combination of behavioral experiments and computational modeling.
- Behavioral Task: Participants engaged in a reversal-learning task where they had to learn and adapt to changing reward probabilities associated with left/right actions. This task was augmented with a flanker task, where irrelevant arrow stimuli (distractors) were presented alongside the target. Trials were either "free" (participants chose their action) or "instructed" (participants followed a specific direction). This design allowed for the manipulation of conflict between external distractors and actions, as well as internal motivational conflicts (instruction vs. subjective value).
- Computational Modeling: A reinforcement learning framework, specifically Q-learning, was used to model participants' behavior. Key parameters were estimated, including:
- Distractor Bias (φ): This parameter captured the tendency to avoid conflict by favoring choices congruent with distractors.
- Learning Rates (α): Separate learning rates were estimated for different conditions (free vs. instructed, congruent vs. incongruent, and for motivational conflicts).
- Inverse Temperature (β): This parameter captured choice stochasticity.
Model comparison was used to determine the best-fitting model and to assess the influence of different conflict types on decision-making and learning.
Datasets
Not discussed in the paper. The study involved data collected from human participants.
Key Insights and Ideas
- Conflict as a Cost: The study provides strong evidence that conflict itself is perceived as a cost. Participants were biased to follow distractor suggestions, especially when their internal evidence for an action was weak, suggesting a trade-off between expected reward and the cost of conflict.
- Motivational Conflict Impedes Learning: When participants had to suppress their preferred action (based on learned values) to follow an instruction (a motivational conflict), their learning rates were significantly reduced. This suggests that the effort to resolve internal conflicts can devalue the learning from outcomes.
- External Conflict's Limited Impact on Learning: Interestingly, conflict arising from external distractors (action vs. distractor) did not appear to significantly impact learning rates, unlike motivational conflicts. This suggests that the source of the conflict is crucial for its effect on learning.
- Choice Matters for Learning: Learning rates were generally higher in "free" choice trials compared to "instructed" trials, suggesting that agency and self-directed learning enhance the learning process.
Key Results
- Decision Stage: Participants showed a "distractor bias," meaning they were more likely to choose actions congruent with irrelevant distractors, especially when their internal value signals were weak. This bias was associated with slower reaction times when participants went against the distractor's suggestion, indicating a conflict cost.
- Learning Stage:
- Learning rates were lower in instructed trials compared to free trials.
- Motivational conflict (instructed-low value trials) led to a significant reduction in learning rates compared to instructed-high value trials.
- Conflict between actions and external distractors did not show a robust effect on learning rates in the best-fitting model.
- Model Performance: The winning computational model (m8) successfully captured these behavioral patterns, incorporating a distractor bias parameter for decisions and separate learning rates for different types of instructed trials.
Limitations and Future Work
- Sample Size: The authors acknowledge that the relatively small sample size (N=20) warrants caution in interpreting some results, particularly correlations.
- Binary Conflict Categorization: The categorization of trials as either "congruent" or "incongruent" might oversimplify the continuous nature of conflict. Future work could explore more nuanced measures.
- Distractor Facilitation vs. Avoidance: The study couldn't definitively distinguish whether distractors facilitated decisions by breaking ties between equally valued options or solely by promoting conflict avoidance. Future studies with neutral distractors could help disentangle these effects.
- Generalizability: While the study demonstrates conflict adaptation effects in a reinforcement learning context, further research is needed to explore the generalizability of these findings across different tasks and populations.
- Neuroimaging: The authors suggest that future studies employing neuroimaging techniques (e.g., M/EEG) could provide deeper insights into the neural mechanisms underlying conflict processing and its integration with decision-making and learning.
Summary updated