Structured chaos shapes spike-response noise entropy in balanced neural networks

Guillaume Lajoie, Jean-Philippe Thivierge, Eric Shea-BrownFrontiers Comput. Neurosci 2014 · OpenReview

TL;DR

This paper investigates how the inherent chaotic dynamics of balanced neural networks affect the variability of their responses to external stimuli. They introduce a novel theoretical framework to bound spike-train entropy, revealing that network-level spike variability is significantly lower than predicted by single-cell measurements, and is strongly influenced by the input signal's statistics.

What Problem Does This Paper Address?

The brain is thought to operate with a delicate balance of excitatory and inhibitory neurons, leading to complex, often chaotic, dynamics. This chaos makes neural systems highly sensitive to initial conditions. A key question in neuroscience is how this inherent neural "noise" impacts the brain's ability to reliably encode external information, like sensory stimuli. Specifically, if a neural network is chaotic, how variable are the patterns of spikes it produces in response to a given input? Existing research has studied variability at the single-cell level or the overall dynamical entropy of networks, but a clear link between these and the joint spike patterns of many neurons in response to a stimulus has been missing. This paper aims to bridge that gap by quantifying the variability of multi-cell spike patterns in chaotic, balanced networks driven by external signals.

Methods

The authors employ a combination of theoretical analysis and numerical simulations.

  • Model: They use a large network of Quadratic Integrate-and-Fire (QIF) neurons, a common model for cortical neurons, connected with sparse, random, balanced excitatory and inhibitory weights. This network is driven by a temporally structured input signal.
  • Random Dynamical Systems (RDS) Theory: A core contribution is the application of RDS theory. This framework allows them to analyze the ensemble behavior of the network (i.e., how many different initial conditions lead to different spike patterns) when driven by a fixed realization of the input signal. This is crucial for understanding variability due to internal network dynamics rather than just input randomness.
  • Symbolic Dynamics and Entropy Calculation: They map the continuous-time phase-space dynamics of the neurons to discrete binary spike trains. This allows them to use information-theoretic measures, specifically spike-response noise entropy, to quantify variability. They derive a theoretical bound for this entropy.
  • Lyapunov Exponents: To characterize the chaotic nature of the system and its scaling properties, they compute Lyapunov exponents, which measure the rate of exponential divergence of nearby trajectories.

Datasets

Not discussed in the paper. The study uses simulated data generated from their neural network model.

Key Insights and Ideas

  • Chaos Doesn't Necessarily Mean High Variability: Despite the network's chaotic nature, the joint spike patterns of many neurons in response to a stimulus are surprisingly less variable than one might expect if each neuron's variability were independent. This is a central finding.
  • Extensive Chaos Scales: The chaotic property of the network, and its implications for variability, scales with network size. This "extensive chaos" means the phenomenon holds even for very large networks, a desirable property for biological systems.
  • Input Statistics Matter: The variability of the network's output is not solely determined by its internal chaotic dynamics. The statistical properties of the input signal—specifically, the balance between the mean input (DC component) and higher-frequency fluctuations—play a significant role in shaping the network's spike-response entropy.
  • Weak Pairwise Correlations Hide Network-Level Effects: While individual pairs of neurons might show very weak and transient correlations, these subtle interactions collectively lead to a significant reduction in overall network spike-train entropy. This highlights the limitations of solely studying pairwise statistics for understanding network behavior.

Key Results

  • Order of Magnitude Reduction in Entropy: The spike-pattern entropy for the entire network is found to be approximately an order of magnitude lower than what would be predicted by extrapolating from single-cell spike-response entropy measurements.
  • Input Modulation of Entropy: By varying the input mean (η) and fluctuation amplitude (ε) while keeping the firing rate constant, they show that network noise entropy can change by almost an order of magnitude. Specifically, entropy increases monotonically as the system transitions from a fluctuation-driven regime to a mean-driven regime.
  • Lyapunov Spectrum is Size-Invariant: The Lyapunov spectrum, a measure of chaos, remains largely invariant with network size, supporting the idea of extensive chaos.

Limitations and Future Work

  • Approximation in Spike Timing: The derivation of the theoretical bound relies on an approximation for when neurons spike based on their phase-space position. While they show this approximation is highly accurate for small time bins, it's still an approximation.
  • Focus on Noise Entropy: The paper quantifies "noise entropy," which reflects variability. Future work could aim to compute "total entropy," which would capture the network's capacity to encode information across different inputs, not just variability for a fixed input. This would involve treating the system more like a stochastic differential equation.
  • Generalizability to Other Models: The authors suggest their framework could be adapted to other neuron models, but this remains a future direction.
  • Experimental Validation: While the model is biologically plausible, direct experimental validation of these specific entropy bounds in biological networks would be a significant next step.

Summary updated

Abstract

p>Large networks of sparsely coupled, excitatory and inhibitory cells occur throughout the brain. For many models of these networks, a striking feature is that their dynamics are chaotic and thus, are sensitive to small perturbations. How does this chaos manifest in the neural code? Specifically, how variable are the spike patterns that such a network produces in response to an input signal? To answer this, we derive a bound for a general measure of variability—spike-train entropy. This leads to important insights on the variability of multi-cell spike pattern distributions in large recurrent networks of spiking neurons responding to fluctuating inputs. The analysis is based on results from random dynamical systems theory and is complemented by detailed numerical simulations. We find that the spike pattern entropy is an order of magnitude lower than what would be extrapolated from single cells. This holds despite the fact that network coupling becomes vanishingly sparse as network size grows—a phenomenon that depends on “extensive chaos,” as previously discovered for balanced networks without stimulus drive. Moreover, we show how spike pattern entropy is controlled by temporal features of the inputs. Our findings provide insight into how neural networks may encode stimuli in the presence of inherently chaotic dynamics.</p>
BibTeX
@inproceedings{lajoie2014structured,
  author = {Guillaume Lajoie and Jean-Philippe Thivierge and Eric Shea-Brown},
  title = {{Structured chaos shapes spike-response noise entropy in balanced neural networks}},
  booktitle = {Frontiers Comput. Neurosci},
  year = {2014},
  url = {https://openreview.net/forum?id=vdRj0IlWRd},
}
Figure