Biologically plausible solutions for spiking networks with efficient coding

Veronika Koren, Stefano Panzeri

TL;DR

This paper derives a biologically realistic spiking neural network model directly from the principles of "efficient coding"—the idea that brains minimize error and energy. By enforcing biological constraints like Dale’s Law and adaptation, the authors demonstrate that complex brain phenomena, such as population oscillations and spontaneous "Up/Down" states, are not just biological quirks but necessary consequences of computational efficiency.

What Problem Does This Paper Address?

For decades, neuroscientists have debated whether the brain's complex dynamics—like the rhythmic firing of neurons or the specific way excitatory and inhibitory cells connect—are "bugs" of biology or "features" of computation.

The theory of efficient coding suggests that neurons fire spikes to represent signals with maximum accuracy using minimum metabolic energy. While mathematically elegant, early efficient coding models were "un-biological": they allowed a single neuron to be both excitatory and inhibitory (violating Dale’s Law) and ignored well-known cellular properties like adaptation or rebound currents. This paper tackles the gap between abstract mathematical optimality and the messy reality of biological circuits, asking: Can we derive a truly biological network from first principles of efficiency?

Methods

The authors start with a mathematical objective: minimize a loss function that combines coding error (how well the network tracks a stimulus) and metabolic cost (how many spikes are used).

Unlike previous models that used a single loss function, the authors propose two separate loss functions—one for excitatory (E) neurons and one for inhibitory (I) neurons. This allows them to analytically derive the subthreshold dynamics of a Generalized Leaky Integrate-and-Fire (LIF) network.

Key technical innovations include:

  • Dale’s Law Enforcement: They mathematically "prune" connections that would force a neuron to be both E and I.
  • Low-Rank Connectivity: They use structured, low-rank matrices to define how neurons interact, constraining activity to a low-dimensional manifold, which is consistent with empirical brain recordings.
  • Parameter Mapping: They show that abstract mathematical terms in their derivation map directly to biological features like fast/slow synaptic currents and local homeostatic currents.

Datasets

As a theoretical and computational study, this paper does not use external experimental datasets (like MNIST or ImageNet). Instead, the authors evaluate their model using:

  • Synthetic Stimuli: Step inputs and multi-dimensional independent features used to test how the network tracks signals.
  • Simulated Neural Populations: Networks of 400 Excitatory and 100 Inhibitory units, a ratio (4:1) commonly found in the mammalian cortex.

Key Insights and Ideas

  • Mathematical Serendipity: Perhaps the most striking insight is that when you derive a network for efficiency, terms for spike-triggered adaptation and rebound currents (h-currents) emerge naturally. These aren't just added for "realism"; the math demands them to maintain efficiency.
  • The Role of E-E Connectivity: In many simple models, excitatory neurons don't connect to each other. Here, the authors show that recurrent E-E connections are vital for "cooperation" between neurons, allowing the network to maintain a memory of the stimulus.
  • Rank as Complexity: The "rank" of the connectivity matrix acts as a dial for behavioral complexity. A rank-1 network simply tracks a stimulus. A higher-rank network begins to "mix" features and can sustain activity even after the stimulus is gone.
  • Oscillations are Efficient: The population-wide oscillations often seen in the brain (like gamma rhythms) emerge in this model as a way to balance the coding of new information with the metabolic cost of firing.

Key Results

  • Biological Realism: The model successfully accounts for approximately 90% of cortical neurons, specifically pyramidal cells (E) and fast-spiking interneurons (I).
  • Emergent Dynamics: The network exhibits spontaneous Up and Down states—periods of high and low activity seen during sleep or anesthesia—purely as a result of the interaction between recurrent excitation and spike-triggered adaptation.
  • Heterogeneity: The derivation results in neurons with different firing thresholds and reset potentials, moving away from the "cookie-cutter" identical neurons found in standard AI models.
  • Fast vs. Slow: The model correctly predicts that inhibitory neurons should have faster membrane time constants than excitatory neurons to maintain stability and efficiency.

Limitations and Future Work

  • Linearity: The current framework is limited to networks performing linear transformations of their inputs. Real-world brain computation is often non-linear.
  • Learning Rules: The connectivity in this paper is derived analytically (pre-calculated). The authors note that future work should explore how these structured connections could be "learned" through biologically plausible plasticity rules (like STDP).
  • Performance Benchmarking: While the model looks biological, more work is needed to quantify exactly how much "efficiency" is lost when moving from an abstract model to this more constrained biological one.

Summary updated

Abstract

Understanding how the dynamics of neural networks is shaped by the computations they perform is a fundamental question in neuroscience. Recently, the framework of efficient coding proposed a theory of how spiking neural networks can compute low-dimensional stimulus signals with high efficiency. Efficient spiking networks are based on time-dependent minimization of a loss function related to information coding with spikes. To inform the understanding of the function and dynamics of biological networks in the brain, however, the mathematical models have to be informed by biology and obey the same constraints as biological networks. Currently, spiking network models of efficient coding have been extended to include some features of biological plausibility, such as architectures with excitatory and inhibitory neurons. However, biological realism of efficient coding theories is still limited to simple cases and does not include single neuron and network properties that are known to be key in biological circuits. Here, we revisit the theory of efficient coding with spikes to develop spiking neural networks that are closer to biological circuits. Namely, we find a biologically plausible spiking model realizing efficient coding in the case of a generalized leaky integrate-and-fire network with excitatory and inhibitory units, equipped with fast and slow synaptic currents, local homeostatic currents such as spike-triggered adaptation, hyperpolarization-activated rebound current, heterogeneous firing thresholds and resets, heterogeneous postsynaptic potentials, and structured, low-rank connectivity. We show how the complexity of E-E connectivity matrix shapes network responses.
BibTeX
@inproceedings{koren2022biologically,
  author = {Veronika Koren and Stefano Panzeri},
  title = {{Biologically plausible solutions for spiking networks with efficient coding}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=zdmYnIRXvKS},
}
Figure