The combination of STDP and intrinsic plasticity yields complex dynamics in recurrent spiking networks

Andreea Lazar, Gordon Pipa, Jochen TrieschEsann 2006

Introduction

In the study of neural computation, Spike-Timing Dependent Plasticity (STDP) is a fundamental rule where "neurons that fire together, wire together." While effective for learning sequences, applying STDP to recurrent networks often leads to instability, where weights polarize to extremes and activity collapses into simple, repetitive loops. This research explores the synergy between STDP and Intrinsic Plasticity (IP)—a homeostatic mechanism that regulates individual neuron excitability. By combining these two rules, the authors demonstrate that recurrent spiking networks can transition from trivial states to highly complex, brain-like dynamics.

Methods

The researchers developed a deterministic recurrent network of $N$ binary spiking units governed by three primary constraints:

  1. k-Winner-Take-All (kWTA): To ensure population sparseness, only the $k$ most excited neurons fire at any given time step, mimicking the role of inhibitory interneurons.
  2. STDP: Synaptic connections are strengthened if the pre-synaptic neuron fires immediately before the post-synaptic one, and weakened if the order is reversed.
  3. Intrinsic Plasticity (IP): Each neuron adjusts its firing threshold based on its history. If a neuron fires too frequently, its threshold increases; if it fires too rarely, the threshold decreases. This ensures "lifetime sparseness," allowing all neurons to participate over time.

The complexity of the resulting dynamics was evaluated using the length and number of limit cycles—the repeating sequences of states that a finite, deterministic system must eventually enter.

Synergistic Plasticity and Network Structure

A key finding is that STDP and IP are synergistic rather than merely additive. While STDP attempts to carve specific activity paths, IP prevents any single group of neurons from dominating the network. This interaction pushes the system into a high-dimensional regime.

Distributions of weights and thresholds in the STDP+IP network.

Unlike many STDP-only models where weights become bimodal (either 0 or 1), the inclusion of IP results in a unimodal weight distribution where most connections remain weak. This structural outcome is significantly closer to observations in biological cortical circuits. Furthermore, a strong correlation emerges between a neuron's threshold and the sum of its incoming weights, indicating a self-organizing balance.

Emergent Dynamics and Complexity

The combination of these plasticity rules dramatically increases the network's state-space coverage and cycle length. For a 100-neuron network, STDP+IP cycles often exceeded $10^5$ time steps, compared to cycles of just 3 steps in STDP-only configurations.

Spiking activity and firing statistics showing Poisson-like dynamics.

Despite the deterministic nature of the model, the "full" system exhibits firing patterns that resemble the stochastic, noisy activity of real brains. The interspike interval (ISI) distribution follows a near-exponential decay, characteristic of Poisson-like firing. This suggests that homeostasis (IP), typically viewed as a stabilizing force, actually serves as a driver of complexity by preventing the network from falling into "lazy" loops.

Conclusion and Future Work

This study highlights how simple local rules can lead to sophisticated global behavior. The authors suggest that these self-organizing dynamics could be ideal for Reservoir Computing, specifically Liquid State Machines. Future research should investigate whether these synergistic networks provide superior performance on computational benchmarks, such as speech recognition or motor control, compared to traditional static reservoirs.

Summary updated

BibTeX
@inproceedings{lazar2006combination,
  author = {Andreea Lazar and Gordon Pipa and Jochen Triesch},
  title = {{The combination of STDP and intrinsic plasticity yields complex dynamics in recurrent spiking networks}},
  booktitle = {Esann},
  year = {2006},
}
Figure