Structure, dynamics, coding and optimal biophysical parameters of efficient excitatory-inhibitory spiking networks

Veronika Koren, Simone Blanco Malerba, Tilo Schwalger, Stefano PanzeriBiorxiv 2024 · OpenReview

The principle of efficient coding posits that sensory cortical networks are designed to encode maximal sensory information with minimal metabolic cost. Despite its influence, it has remained unclear whether fundamental empirical properties of neural network activity can be explained solely based on this normative principle. This research provides a rigorous mathematical bridge between high-level efficiency principles and the low-level biophysics of the brain, deriving the structure of a spiking neural network from first principles.

The Efficient Coding Hypothesis

The brain has evolved to represent the world accurately using the least amount of energy, typically measured in spikes. While this theory is well-established, testing it often involves abstract models that ignore physical constraints like Dale’s Law—the rule that a neuron is either excitatory or inhibitory. By incorporating these biological realities, the authors investigated whether specific structural properties of the cortex, such as the disproportionate number of excitatory versus inhibitory neurons, are necessary for efficiency rather than biological accidents.

Deriving Biology from Mathematics

The researchers used a normative approach to derive a network that satisfies a specific mathematical objective. They defined a loss function balancing encoding error (tracking a time-varying stimulus) and metabolic cost (spike count). From this, they derived a "greedy spiking rule" where a neuron fires only if it reduces the total loss.

This derivation naturally results in a generalized Leaky Integrate-and-Fire (gLIF) model. To respect Dale’s Law, the authors applied rectification to the connectivity matrices, ensuring distinct excitatory (E) and inhibitory (I) populations. The model was tested using synthetic stimulus features modeled as stochastic Ornstein-Uhlenbeck processes, representing continuous, time-varying signals.

Key Findings: The 4:1 Ratio and Feature-Specific Competition

The study's most striking result is that the 4:1 E-I ratio found in the rodent cortex emerged as the mathematically optimal ratio for minimizing the network's loss function. Additionally, the model predicted an I-I to E-I synaptic strength ratio of 3:1, closely matching empirical cortical measurements.

To maintain efficiency, the network implements feature-specific competition. Neurons with similar stimulus tuning inhibit each other to prevent redundant spiking. This "like-to-like" inhibition aligns with recent optogenetic findings in the visual cortex. Furthermore, the network maintains a "tight instantaneous E-I balance," where excitatory and inhibitory inputs cancel each other out in real-time, a key predictor of coding efficiency.

Performance and Spiking Dynamics

The derived network achieved near-perfect tracking of complex, time-varying signals, with $R^2$ values between 0.95 and 0.97. Unlike simpler models, this E-I architecture proved highly robust, maintaining realistic firing rates even under varying metabolic constraints.

Tracking performance and spiking dynamics of the efficient E-I network.

As shown in the figure above, the network tracks multi-dimensional signals (dim. 1-3) with high precision. The raster plots illustrate the coordinated activity of excitatory and inhibitory populations, while the rate plots demonstrate the balanced dynamics required for high-fidelity signal representation.

Limitations and Future Outlook

While the model successfully "post-dicts" many biological features, it currently lacks recurrent excitatory-to-excitatory (E-E) connections, which are crucial for memory and signal amplification in the brain. Additionally, the optimal parameters depend on the relative weighting of error versus cost, which can vary across different biological systems. Future work aims to explore how synaptic plasticity rules, such as Spike-Timing-Dependent Plasticity (STDP), might allow these efficient structures to be learned from experience.

Summary updated

Abstract

The principle of efficient coding posits that sensory cortical networks are designed to encode maximal sensory information with minimal metabolic cost. Despite the major influence of efficient coding in neuro-science, it has remained unclear whether fundamental empirical properties of neural network activity can be explained solely based on this normative principle. Here, we rigorously derive the structural, coding, biophysical and dynamical properties of excitatory-inhibitory recurrent networks of spiking neurons that emerge directly from imposing that the network minimizes an instantaneous loss function and a time-averaged performance measure enacting efficient coding. The optimal network has biologically-plausible biophysical features, including realistic integrate-and-fire spiking dynamics, spike-triggered adaptation, and a non-stimulus-specific excitatory external input regulating metabolic cost. The efficient network has excitatory-inhibitory recurrent connectivity between neurons with similar stimulus tuning implementing feature-specific competition, similar to that recently found in visual cortex. Networks with unstructured connectivity cannot reach comparable levels of coding efficiency. The optimal biophysical parameters include 4 to 1 ratio of excitatory vs inhibitory neurons and 3 to 1 ratio of mean inhibitory-to-inhibitory vs. excitatory-to-inhibitory connectivity that closely match those of cortical sensory networks. The efficient network has biologically-plausible spiking dynamics, with a tight instantaneous E-I balance that makes them capable to achieve efficient coding of external stimuli varying over multiple time scales. Together, these results explain how efficient coding may be implemented in cortical networks and suggests that key properties of biological neural networks may be accounted for by efficient coding.
BibTeX
@misc{koren2024structure,
  author = {Veronika Koren and Simone Blanco Malerba and Tilo Schwalger and Stefano Panzeri},
  title = {{Structure, dynamics, coding and optimal biophysical parameters of efficient excitatory-inhibitory spiking networks}},
  year = {2024},
  url = {https://openreview.net/forum?id=Y0ry1DaO0d},
  howpublished = {Biorxiv},
}
Figure