Temporally Asymmetric Hebbian Learning, Spike liming and Neural Response Variability

L. F. Abbott, Sen SongNeurIPS 1998 · OpenReview

Abstract

Recent experimental data indicate that the strengthening or weakening of synaptic connections between neurons depends on the relative timing of pre- and postsynaptic action potentials. This paper demonstrates that a biologically realistic learning rule based on precise spike timing naturally drives neurons into a "balanced state." In this state, excitatory and inhibitory inputs cancel each other out, leading to the highly irregular, unpredictable firing patterns observed in real living brains.

Introduction: The Mystery of Irregular Firing

In classical artificial neural networks, "learning" usually involves adjusting weights based on continuous firing rates. However, real biological neurons communicate via discrete electrical pulses called spikes. A long-standing mystery in neuroscience is why neurons in a living brain fire so irregularly—looking more like a random Poisson process than a steady clock.

To explain this, researchers previously proposed the "balanced excitation-inhibition" model. In this model, a neuron receives massive amounts of both "push" (excitatory) and "pull" (inhibitory) signals that nearly cancel out. The neuron only fires when a random fluctuation briefly pushes it over the threshold. While this model matches experimental data, it usually requires researchers to manually "fine-tune" the synaptic weights to achieve that perfect balance. The central question addressed by Abbott and Song is: Can a local learning rule achieve this balance automatically?

Computational Methods

The authors employ a computational model of an integrate-and-fire neuron. This model simulates a cell receiving 1,000 excitatory and 200 inhibitory inputs, each firing as independent Poisson spike trains based on parameters derived from landmark electrophysiology experiments.

The core of their approach is the Temporally Asymmetric Hebbian Learning rule (now commonly known as Spike-Timing-Dependent Plasticity, or STDP). Unlike traditional Hebbian rules ("cells that fire together, wire together"), this rule is sensitive to the order of spikes.

The Temporally Asymmetric Hebbian Learning rule (STDP).

As illustrated in the learning rule:

  • LTP (Long-Term Potentiation): If an input spike arrives just before the neuron fires, the connection is strengthened, reflecting causality.
  • LTD (Long-Term Depression): If an input spike arrives just after the neuron fires, the connection is weakened.

A crucial methodological assumption made by the authors is that the "weakening" (LTD) side of the rule is slightly more powerful than the "strengthening" (LTP) side.

Mechanisms of Self-Organization

By making the area under the LTD curve larger than the LTP curve, the authors created a built-in "brake" for the system. If a neuron fires randomly (without a causal link to its inputs), its connections will, on average, weaken. This prevents the "runaway excitation" often seen in simpler Hebbian models.

The most profound insight is that the STDP rule acts as a feedback loop that seeks out the balanced state. If the excitatory weights are too strong, the neuron fires regularly and "randomly" relative to its inputs, causing the LTD-heavy rule to trim the weights down. Once the weights are weak enough that the neuron only fires when inputs happen to coincide, the LTP/LTD balance stabilizes. Furthermore, the rule is indifferent to the rate of input firing but highly sensitive to synchrony. If a group of inputs fire at the same time, they are more likely to trigger a postsynaptic spike, marking them for strengthening.

Results: Transition to Irregularity

The model successfully demonstrates a transition from a regular firing mode to an irregular firing mode. The system reaches a stable equilibrium where the asymmetry of the learning rule is perfectly matched by the temporal correlation of the spikes.

Irregular firing pattern characteristic of the balanced state.

This self-balancing behavior, resulting in the highly variable spike timing shown above, was found to be robust across a wide range of input firing rates (from 5 Hz to 25 Hz). The network naturally tunes itself to a state where the Coefficient of Variation (CV) approaches 1, mimicking in vivo observations.

Limitations and Future Work

While groundbreaking, the study has a few notable limitations:

  • The LTD Assumption: The conclusion relies heavily on the assumption that LTD is stronger than LTP overall. While experimental data at the time was suggestive, it wasn't definitive.
  • Fixed Inhibition: In this study, only excitatory synapses were modified. In a real brain, inhibitory synapses are also plastic, and future models should explore how inhibitory plasticity interacts with this system.
  • Simplified Neurons: The integrate-and-fire model is a "point neuron"—it ignores the complex dendritic tree structures of real neurons, which likely play a massive role in how spike timing is processed locally.

Summary updated

Abstract

Recent experimental data indicate that the strengthening or weakening of synaptic connections between neurons depends on the relative timing of pre- and postsynaptic action potentials. A Hebbian synaptic modification rule based on these data leads to a stable state in which the excitatory and inhibitory inputs to a neuron are balanced, producing an irregular pattern of firing. It has been proposed that neurons in vivo operate in such a mode.
BibTeX
@inproceedings{abbott1998temporally,
  author = {L. F. Abbott and Sen Song},
  title = {{Temporally Asymmetric Hebbian Learning, Spike liming and Neural Response Variability}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1998},
  url = {https://openreview.net/forum?id=ryg0nv-OWr},
}
Figure