Control of criticality and computation in spiking neuromorphic networks with plasticity

Benjamin Cramer, David Stöckel, Markus Kreft, Michael Wibral, Johannes Schemmel, Karlheinz Meier, Viola PriesemannarXiv 2019 · DOI · arXiv

TL;DR

This paper challenges the long-held dogma that "criticality"—operating at the boundary between order and chaos—is universally optimal for computation in neural networks. By implementing a plastic, spiking neural network on a physical neuromorphic chip, the researchers demonstrate that while criticality maximizes abstract information capacity, actual task performance depends heavily on task complexity: complex, memory-intensive tasks thrive at criticality, whereas simpler tasks actually suffer from it.

What Problem Does This Paper Address?

In recurrent neural network (RNN) design, a popular guiding principle is to tune the network to a "critical state" or the "edge of chaos." At this transition point, abstract computational properties like dynamic range, sensitivity, and information transfer are maximized. Because of this, researchers have widely assumed that criticality is a silver bullet for maximizing task performance.

However, the authors of this paper question this blanket assumption. Operating at criticality causes a network to have a very long memory (long correlation times). While this is great for tracking long-term sequences, what happens if a task only requires short-term information? In that case, a long memory might cause the network to retain task-irrelevant "garbage" from the past, cluttering its state and degrading performance. The authors set out to rigorously test how distance to criticality impacts performance across tasks of varying complexity.

Methods

The researchers emulated a network of 32 leaky integrate-and-fire (LIF) neurons on the BrainScaleS 2 neuromorphic prototype system—a physical piece of hardware that runs analog neural simulations.

The BrainScaleS 2 neuromorphic system and chip alongside spike rasters showing critical (low input) vs. asynchronous (high input) dynamics.

Key aspects of their methodology include:

  • Local Plasticity: The chip features an on-chip processor that executes local learning rules combining a positive weight drift with an anti-causal Spike-Timing-Dependent Plasticity (STDP) rule, achieving stable homeostatic regulation.
  • Tuning Criticality via Input: Instead of fine-tuning internal network parameters, the authors altered the network's state simply by changing the external input strength ($K_{ext}$). Low input strength allowed internal recurrent connections to dominate, driving the network to self-organize toward criticality. High input strength forced the network into a subcritical, asynchronous-irregular state.
  • Analysis Frameworks: They verified criticality using classical avalanche distributions and branching ratios, and evaluated information processing using advanced information-theoretic tools like Partial Information Decomposition (PID).

Datasets

Because this study focuses on fundamental computational properties rather than real-world pattern recognition, the authors evaluated the network using synthetic benchmark tasks within a reservoir computing framework:

  • $n$-bit Sum Task: A linear task where the network calculates the sum of inputs over the last $n$ time steps.
  • $n$-bit Parity Task: A highly non-linear task where the network computes the parity (odd or even) of the inputs over the last $n$ time steps.

By varying $n$ (from 5 to 25 bits), they precisely controlled the memory depth and complexity required to solve the task.

Key Insights and Ideas

  • The Criticality Dogma Busted: Criticality is not universally optimal. For simple tasks (low $n$), the network performs best when it is subcritical. This is because subcritical networks exhibit "fast-forgetting," which actively helps clear out old, task-irrelevant information.
  • Capacity $\neq$ Performance: Information-theoretic measures (like active information storage and transfer entropy) confirm that the network's raw capacity peaks at criticality. However, raw capacity does not automatically translate to task success; the network state must match the task's specific demands.
  • Dynamic Task Adaptation: Because the network's state is controlled purely by input strength, a system can be dynamically toggled between critical and subcritical states in a fraction of a second to adapt to changing task complexities.

Key Results

  • Task Performance Splits: For complex, memory-heavy tasks ($n = 25$), performance peaked sharply near criticality (low input strength). For simple tasks ($n = 5$), performance peaked in the subcritical regime (high input strength).
Task performance on $n$-bit sum and parity tasks across different input strengths, demonstrating that complex tasks prefer criticality while simple tasks prefer subcritical states.
  • Hardware Acceleration: Emulating the network on the BrainScaleS 2 neuromorphic chip yielded a 100x speedup compared to standard software simulations (using the Brian 2 simulator). A 600-second biological simulation took just 6 seconds on the chip.
  • Avalanche Scaling: Software controls confirmed that the network's avalanche size distributions follow a power-law that scales with system size, a definitive hallmark of true physical criticality.

Limitations and Future Work

The primary limitation of this study is the scale of the prototype hardware, which was limited to just 32 neurons. While the authors used software simulations to show that their findings scale up to larger networks, physical validation on larger chips is still required. The authors note that future iterations of the BrainScaleS 2 chip will support 512 neurons, which will allow for more complex, large-scale emulations. Additionally, the authors had to omit the causal arm of STDP because it destabilized the network, suggesting that more complex multi-scale plasticity rules may be needed to maintain stability in larger physical systems.

Summary updated

Abstract

The critical state is assumed to be optimal for any computation in recurrent neural networks, because criticality maximizes a number of abstract computational properties. We challenge this assumption by evaluating the performance of a spiking recurrent neural network on a set of tasks of varying complexity at - and away from critical network dynamics. To that end, we developed a spiking network with synaptic plasticity on a neuromorphic chip. We show that the distance to criticality can be easily adapted by changing the input strength, and then demonstrate a clear relation between criticality, task-performance and information-theoretic fingerprint. Whereas the information-theoretic measures all show that network capacity is maximal at criticality, this is not the case for performance on specific tasks: Only the complex, memory-intensive tasks profit from criticality, whereas the simple tasks suffer from it. Thereby, we challenge the general assumption that criticality would be beneficial for any task, and provide instead an understanding of how the collective network state should be tuned to task requirement to achieve optimal performance.
BibTeX
@misc{cramer2019control,
  author = {Benjamin Cramer and David Stöckel and Markus Kreft and Michael Wibral and Johannes Schemmel and Karlheinz Meier and Viola Priesemann},
  title = {{Control of criticality and computation in spiking neuromorphic networks with plasticity}},
  year = {2019},
  eprint = {1909.08418},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.08418},
}
Figure