Reinforcement Learning With Modulated Spike Timing–Dependent Synaptic Plasticity

Michael A. Farries, Adrienne L. FairhallJournal of Neurophysiology 2007 · DOI

Abstract

Spike timing-dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We also describe in detail how this model might be implemented biologically. Thus our model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output.
BibTeX
@article{farries2007reinforcement,
  author = {Michael A. Farries and Adrienne L. Fairhall},
  title = {{Reinforcement Learning With Modulated Spike Timing–Dependent Synaptic Plasticity}},
  journal = {Journal of Neurophysiology},
  year = {2007},
  doi = {10.1152/jn.00364.2007},
}

Loading related content...

Figure