Neural heterogeneity controls computations in spiking neural networks

Richard Gast, Sara A. Solla, Ann KennedyProceedings of the National Academy of Sciences 2024 · DOI

TL;DR

This paper explores how variations in neuron properties (specifically spike thresholds) within the same cell type influence neural network computations. They found that this "within-type heterogeneity" allows inhibitory neurons to precisely gate signals and excitatory neurons to better process information, suggesting a flexible mechanism for tuning neural circuits for different tasks.

What Problem Does This Paper Address?

The brain is a marvel of complex computation, built from billions of neurons. While we know neurons come in different "types" (like excitatory and inhibitory), it's also clear that even neurons of the same type aren't identical. They have variations in their electrical properties, like their "spike threshold" – the voltage a neuron needs to reach before it fires an action potential. The big question this paper tackles is: how do these subtle, within-type differences in neurons impact the overall function and computational capabilities of neural networks? Do they just add noise, or do they play a crucial, functional role in how the brain processes information, gates signals, and learns?

Methods

The researchers used a mean-field model of spiking neural networks (SNNs). This approach simplifies the complex interactions of individual neurons by describing the average behavior of a large population. They specifically focused on spike threshold heterogeneity, introducing variations in this parameter across neurons within a population. They derived analytical mean-field equations that allowed them to systematically study how different degrees of heterogeneity (quantified by a parameter $\Delta_v$) affected network dynamics. To validate their model, they also performed numerical simulations of SNNs with sparse connectivity, comparing the results to their analytical predictions. They analyzed network dynamics by looking at bifurcation diagrams (which map out different dynamic states as parameters change) and assessed computational functions like signal gating, encoding, and decoding using techniques inspired by dynamical systems theory and machine learning (including reservoir computing).

Datasets

Not discussed in the paper. The study relies on simulated neural networks and theoretical modeling, not empirical datasets.

Key Insights and Ideas

  • Heterogeneity as a Computational Knob: The core idea is that within-type heterogeneity isn't just biological noise; it's a tunable parameter that the brain can use to sculpt neural computations.
  • Differentiated Roles for Inhibitory vs. Excitatory Heterogeneity: The paper reveals distinct computational roles for heterogeneity in different neuron types. Inhibitory heterogeneity is key for precise gating, while excitatory heterogeneity enhances encoding and decoding.
  • Inhibitory Heterogeneity for Fine-Tuning Gating: Homogeneous inhibitory interneurons tend to impose synchronized dynamics, narrowing the excitatory population's repertoire. In contrast, heterogeneous inhibitory interneurons act more like an "inhibitory offset," preserving excitatory function while allowing for more nuanced control over signal propagation.
  • Excitatory Heterogeneity for Richer Dynamics: Heterogeneity in excitatory neurons increases the dimensionality of network dynamics, making them better at decoding tasks. This suggests that a diverse excitatory population can represent more complex information.
  • Trade-offs in Encoding: Homogeneous excitatory networks excel at encoding signals through stable, multistable dynamic regimes (useful for memory), but heterogeneous networks are less suited for this specific task due to a reduced bistable regime.

Key Results

  • Gating Control: Varying the spike threshold heterogeneity of inhibitory interneurons allows them to precisely gate signals. Low heterogeneity leads to synchronized dynamics, while higher heterogeneity preserves the excitatory population's dynamic repertoire.
  • Encoding Impairment: In a ring network model for spatial memory, increased excitatory heterogeneity led to a loss of spatially localized activity and impaired encoding of stimulus location. Homogeneous networks showed better retention of this activity.
  • Enhanced Function Generation: Conversely, heterogeneous excitatory networks demonstrated superior capacity for function generation (e.g., producing specific time-dependent output patterns) compared to homogeneous networks, especially in oscillatory regimes. This was linked to higher-dimensional dynamics and reduced trial-to-trial variability.
  • Entrainment Properties: Heterogeneous networks showed increased sensitivity to periodic forcing, allowing them to be entrained over a wider range of frequencies. This is crucial for reliable function generation in oscillating networks.

Limitations and Future Work

  • Model Simplifications: The study primarily uses a mean-field model, which averages over individual neuron behavior. While validated with SNN simulations, it's a simplification of biological reality. The use of a Lorentzian distribution for spike thresholds, while analytically convenient, might not capture all biological distributions.
  • Focus on Spike Thresholds: The research focuses specifically on spike threshold heterogeneity. Future work could explore the impact of heterogeneity in other neuron parameters (e.g., membrane capacitance, time constants) or synaptic properties.
  • Generalizability of Distributions: While they tested Gaussian vs. Lorentzian distributions, it's noted that the robustness of their findings to other non-symmetric or multi-modal distributions remains an open question.
  • Biological Mechanisms: The paper proposes heterogeneity as a mechanism for tuning circuits but doesn't delve deeply into the biological mechanisms (developmental, neuromodulatory) that might establish and maintain this heterogeneity.
  • Future Directions: The authors suggest exploring other types of distributions for heterogeneity and investigating the interplay of heterogeneity with other network properties like synaptic plasticity and connectivity structure. They also highlight the need to understand how this heterogeneity is achieved and maintained in biological systems.

Summary updated

Abstract

The brain is composed of complex networks of interacting neurons that express considerable heterogeneity in their physiology and spiking characteristics. How does this neural heterogeneity influence macroscopic neural dynamics, and how might it contribute to neural computation? In this work, we use a mean-field model to investigate computation in heterogeneous neural networks, by studying how the heterogeneity of cell spiking thresholds affects three key computational functions of a neural population: the gating, encoding, and decoding of neural signals. Our results suggest that heterogeneity serves different computational functions in different cell types. In inhibitory interneurons, varying the degree of spike threshold heterogeneity allows them to gate the propagation of neural signals in a reciprocally coupled excitatory population. Whereas homogeneous interneurons impose synchronized dynamics that narrow the dynamic repertoire of the excitatory neurons, heterogeneous interneurons act as an inhibitory offset while preserving excitatory neuron function. Spike threshold heterogeneity also controls the entrainment properties of neural networks to periodic input, thus affecting the temporal gating of synaptic inputs. Among excitatory neurons, heterogeneity increases the dimensionality of neural dynamics, improving the network's capacity to perform decoding tasks. Conversely, homogeneous networks suffer in their capacity for function generation, but excel at encoding signals via multistable dynamic regimes. Drawing from these findings, we propose intra-cell-type heterogeneity as a mechanism for sculpting the computational properties of local circuits of excitatory and inhibitory spiking neurons, permitting the same canonical microcircuit to be tuned for diverse computational tasks.
BibTeX
@inproceedings{gast2024neural,
  author = {Richard Gast and Sara A. Solla and Ann Kennedy},
  title = {{Neural heterogeneity controls computations in spiking neural networks}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2024},
  doi = {10.1073/pnas.2311885121},
}
Figure