Introduction
Learning in recurrent spiking neural networks (RSNNs) presents a significant challenge in computational neuroscience, particularly when hidden neurons—those not directly receiving sensory input—are involved. Traditional machine learning methods like Backpropagation Through Time (BPTT) are biologically implausible as they require non-local information and backward-in-time signaling. Conversely, standard biological rules like Spike-Timing Dependent Plasticity (STDP) often fail to capture long-term temporal structures or the "hidden causes" of sensory data.
Rezende and Gerstner (2014) address this by deriving a biologically plausible learning rule that combines variational inference with reinforcement learning principles. Their approach allows RSNNs to learn complex spatio-temporal patterns through a local STDP rule modulated by a global "novelty" signal.
Methods: Variational Learning in Spiking Networks
The authors treat the spiking network as a generative model using a Spike Response Model (SRM), where neurons fire stochastically based on their membrane potential. To handle the mathematical intractability of calculating the exact "cause" of a spike train, they employ variational learning.
The framework utilizes an auxiliary "recognition" network (the Q-network) to approximate the posterior distribution of hidden neuron activity. By minimizing the Helmholtz Free Energy, the authors derive a three-factor learning rule where synaptic changes depend on:
- Pre-synaptic activity: A trace of past spikes.
- Post-synaptic state: Voltage and timing.
- Global Novelty Signal ($e_N$): A neuromodulator-like signal indicating how "surprising" the input is relative to the network's internal model.
To stabilize learning in large stochastic networks, they introduced a "baseline removal" trick, subtracting a moving average of the free energy to reduce gradient noise.
Experimental Evaluation
The model was tested on two primary tasks:
- Stair Patterns: A sequence of spike bursts testing the network's ability to learn transition probabilities and long-term dependencies.
- Virtual Maze Task: A simulated rat navigating a 16-room maze, where each room is represented by an MNIST digit. The network learned the maze topology based on sequences of these images.
Key Results
- Necessity of Hidden Neurons: Networks without hidden neurons failed to learn complex patterns, whereas those with hidden neurons captured the data distribution effectively.
- Computational Efficiency: The variance-reduced learning rule was over 100 times faster and more stable than naive stochastic gradient descent.
- Novelty Detection: In the maze task, a change in layout triggered a massive burst in the global error signal ($e_N$), providing a testable prediction for biological neuromodulator activity (e.g., Dopamine or Acetylcholine).
Discussion and Insights
A central insight of this work is the role of "surprise" as a driver for plasticity. The global signal $e_N$ acts as a gate; if the network predicts the input perfectly, no learning occurs. Furthermore, the model suggests a functional duality in synapses, potentially managing weights for both generation ($M$) and inference ($Q$).
While the theory provides a robust bridge between Bayesian inference and biological plasticity, scaling to millions of neurons remains a challenge due to increasing noise. Future work involves experimental validation of the predicted novelty signals in animal models and exploring the biological implementation of the dual $M$ and $Q$ weights, perhaps through distinct neuromodulatory states like wakefulness and sleep.
Reference: Rezende, D. J., & Gerstner, W. (2014). Stochastic variational learning in recurrent spiking networks. Frontiers in Computational Neuroscience, 8, 38.
Summary updated