Supervised learning in spiking neural networks with FORCE training

Wilten Nicola, Claudia ClopathNature Communications 2017 · DOI

Abstract

Populations of neurons display an extraordinary diversity in the behaviors they affect and display. Machine learning techniques have recently emerged that allow us to create networks of model neurons that display behaviors of similar complexity. Here we demonstrate the direct applicability of one such technique, the FORCE method, to spiking neural networks. We train these networks to mimic dynamical systems, classify inputs, and store discrete sequences that correspond to the notes of a song. Finally, we use FORCE training to create two biologically motivated model circuits. One is inspired by the zebra finch and successfully reproduces songbird singing. The second network is motivated by the hippocampus and is trained to store and replay a movie scene. FORCE trained networks reproduce behaviors comparable in complexity to their inspired circuits and yield information not easily obtainable with other techniques, such as behavioral responses to pharmacological manipulations and spike timing statistics.

Introduction

In the quest to understand how the brain learns, researchers often use "top-down" approaches to train artificial networks to perform biological tasks. However, most existing methods for spiking neural networks (SNNs) require tasks to be defined as closed-form differential equations, limiting the complexity of modeled behaviors. Furthermore, many training methods are model-specific and struggle with the discontinuous, non-linear nature of spikes when scaling to high-dimensional temporal signals like birdsong or episodic memory.

Researchers have successfully adapted the FORCE (First-Order Reduced and Controlled Error) method—a powerful training technique originally designed for rate-based neural networks—to train biologically realistic SNNs. By leveraging initial chaotic dynamics, they trained SNNs to perform complex tasks ranging from mimicking chaotic attractors to replaying musical sequences and movie scenes with high precision.

Methods: Adapting FORCE for Spikes

The study demonstrates that the FORCE method can be applied directly to SNNs by using Recursive Least Squares (RLS) to update the network's decoders online. This process minimizes the squared error between the network's output and a "teacher" signal. The synaptic weights are structured as a combination of a static, "backbone" matrix that induces a chaotic state and a learned feedback matrix.

Evolution of neural firing rates during the FORCE training process.

To prove robustness, the method was tested on three popular neuron models: the Theta model, Leaky Integrate-and-Fire (LIF), and the Izhikevich model. For long or complex sequences, the authors introduced an "internal clock" mechanism called High-Dimensional Temporal Signals (HDTS). By partitioning neurons into temporal assemblies that fire at specific intervals, the network can maintain its position during long replays, such as an 8-second movie clip.

Key Insights and Biological Mimicry

  • Chaos as a Resource: Initializing networks in a chaotic regime provides a "reservoir" of diverse dynamics that the FORCE method "tames" and reshapes into the desired output.
  • Adaptation and Memory: The Izhikevich model, which includes spike-frequency adaptation, performed best on long signals, suggesting that slower biological time scales are crucial for memory capacity.
  • Temporal Scaffolding: Without HDTS, networks struggled with long sequences. HDTS acts like a scaffold, similar to the "time cells" discovered in the hippocampus.
  • Pharmacological Mimicry: Remarkably, when the authors manipulated a trained "songbird" network (e.g., upscaling excitatory weights), the resulting "seizure-like" vocalizations matched experimental results from real birds injected with bicuculline.

Key Results

The FORCE-trained SNNs demonstrated exceptional performance across various tasks:

  • High Fidelity Replay: The networks replayed Beethoven’s Ode to Joy with 82% accuracy and a movie scene with a correlation coefficient of r = 0.98.
  • Robustness: The movie replay network remained functional even after a 50% reduction in neurons, showing high resilience to "lesions."
  • Generalization: Once trained on the Lorenz system, the SNN could reproduce the famous "butterfly" attractor even after the teacher signal was deactivated.
High-fidelity tracking of the teaching signal by the trained network output.

Limitations and Future Work

While the resulting SNNs are biologically realistic and produce realistic spike-timing statistics, the RLS algorithm itself is not fully biologically plausible. It requires non-local information that real synapses likely cannot access. Finding a "local" version of FORCE is a critical next step. Additionally, while the authors demonstrated that biological constraints like Dale’s Law could be enforced, scaling these simulations from thousands to millions of neurons remains a significant computational challenge.

Summary updated

Abstract

Populations of neurons display an extraordinary diversity in the behaviors they affect and display. Machine learning techniques have recently emerged that allow us to create networks of model neurons that display behaviors of similar complexity. Here we demonstrate the direct applicability of one such technique, the FORCE method, to spiking neural networks. We train these networks to mimic dynamical systems, classify inputs, and store discrete sequences that correspond to the notes of a song. Finally, we use FORCE training to create two biologically motivated model circuits. One is inspired by the zebra finch and successfully reproduces songbird singing. The second network is motivated by the hippocampus and is trained to store and replay a movie scene. FORCE trained networks reproduce behaviors comparable in complexity to their inspired circuits and yield information not easily obtainable with other techniques, such as behavioral responses to pharmacological manipulations and spike timing statistics.
BibTeX
@inproceedings{nicola2017supervised,
  author = {Wilten Nicola and Claudia Clopath},
  title = {{Supervised learning in spiking neural networks with FORCE training}},
  booktitle = {Nature Communications},
  year = {2017},
  doi = {10.1038/s41467-017-01827-3},
}
Figure