The functional role of sequentially neuromodulated synaptic plasticity in behavioural learning

Grace Wan Yu Ang, Clara S. Tang, Y. Audrey Hay, Sara Zannone, Ole Paulsen, Claudia ClopathPLOS Computational Biology 2021 · DOI

Abstract

To survive, animals have to quickly modify their behaviour when the reward changes. The internal representations responsible for this are updated through synaptic weight changes, mediated by certain neuromodulators conveying feedback from the environment. In previous experiments, we discovered a form of hippocampal Spike-Timing-Dependent-Plasticity (STDP) that is sequentially modulated by acetylcholine and dopamine. Acetylcholine facilitates synaptic depression, while dopamine retroactively converts the depression into potentiation. When these experimental findings were implemented as a learning rule in a computational model, our simulations showed that cholinergic-facilitated depression is important for reversal learning. In the present study, we tested the model's prediction by optogenetically inactivating cholinergic neurons in mice during a hippocampus-dependent spatial learning task with changing rewards. We found that reversal learning, but not initial place learning, was impaired, verifying our computational prediction that acetylcholine-modulated plasticity promotes the unlearning of old reward locations. Further, differences in neuromodulator concentrations in the model captured mouse-by-mouse performance variability in the optogenetic experiments. Our line of work sheds light on how neuromodulators enable the learning of new contingencies.

Introduction

In a dynamic environment, survival hinges on "reversal learning"—the capacity to abandon a previously rewarded behavior in favor of a new, more beneficial one. While synaptic plasticity—the modification of connection strengths between neurons—is recognized as the fundamental mechanism of learning, traditional models like Hebbian learning ("cells that fire together, wire together") struggle to explain how existing connections are efficiently "undone" when they lose their utility.

Recent research identified a mechanism termed sequentially neuromodulated synaptic plasticity (sn-Plast). This process involves acetylcholine (ACh) facilitating the weakening (depression) of synapses, while dopamine (DA) can retroactively transform that weakening into strengthening (potentiation) if a reward follows. This study aims to bridge the gap between these cellular observations and behavioral outcomes, investigating whether this specific chemical sequence governs how animals navigate and adapt to shifting reward contingencies.

Methods

The study integrated experimental biology with computational modeling to test the functional role of sn-Plast.

Experimental Design

Researchers utilized transgenic mice (ChAT-ArchT) and implanted optic fibers to selectively silence cholinergic neurons in the medial septum via optogenetics. The behavioral task took place in a circular arena where mice first learned to locate a food reward (Initial Learning). Once established, the reward was relocated to the opposite side of the arena (Reversal Learning).

The study analyzed data from three groups:

  • Light-on (ACh-suppressed): $n=21$
  • Light-off (Control): $n=16$
  • GFP-control (Light-on, no silencing construct): $n=8$

Computational Modeling

A spiking neural network (SNN) was developed, connecting "place cells" (spatial representation) to "action cells" (movement). The sn-Plast rule was implemented, gating synaptic changes based on ACh and DA levels. The model was then used to simulate ACh silencing to compare against live behavioral data and extract individual learning parameters.

Results

The experimental results confirmed a selective impairment in learning flexibility. During the initial learning phase, silencing ACh neurons had no significant effect; all groups reached an 80% success rate within approximately 6-7 days. However, a stark deficit emerged during reversal learning. While control mice adapted quickly to the new reward location, ACh-suppressed mice were significantly impaired, with several failing to reach the success threshold even after 12 days of training.

Behavioral task and learning performance. (A) Spatial navigation task setup. (B) Success rates over time for initial and reversal learning. (C) Days required to reach 80% success threshold.

The computational model accurately replicated these behavioral curves. Simulations revealed that reducing the ACh parameter caused the agent to persist in visiting the original reward location, as it could not weaken the established "place-action" synapses rapidly enough. This suggests that ACh acts as a synaptic "reset" or "unlearning" signal, weakening active synapses during exploration to clear out associations that no longer yield rewards.

Discussion

These findings highlight that learning is not merely a product of chemical presence, but of chemical sequence. ACh sets the stage by depressing synapses; if a reward (signaled by DA) follows, that depression is flipped to potentiation. Without the reward, the synapse remains weak, facilitating the extinction of the old behavior. Furthermore, the study suggests that individual variability in learning speed—"fast" vs. "slow" learners—may be attributed to differences in neuromodulatory "tuning" (sensitivity to ACh and DA).

Limitations and Future Directions

The current SNN is a minimal model and does not yet incorporate factors such as memory consolidation during sleep or fluctuating motivation levels. Additionally, while the model predicts that ACh/DA levels explain individual differences, real-time measurement of these chemicals in vivo using fluorescent biosensors is a necessary next step to confirm these predictions.

Summary updated

BibTeX
@article{ang2021functional,
  author = {Grace Wan Yu Ang and Clara S. Tang and Y. Audrey Hay and Sara Zannone and Ole Paulsen and Claudia Clopath},
  title = {{The functional role of sequentially neuromodulated synaptic plasticity in behavioural learning}},
  journal = {PLOS Computational Biology},
  year = {2021},
  doi = {10.1371/journal.pcbi.1009017},
}
Figure