Dimensionality in recurrent spiking networks: global trends in activity and local origins in connectivity

Stefano Recanatesi, Gabriel Koch Ocker, Michael A. Buice, Eric Shea‐Brown

Abstract

The dimensionality of a network’s collective activity is of increasing interest in neuroscience. This is because dimensionality provides a compact measure of how coordinated network-wide activity is, in terms of the number of modes (or degrees of freedom) that it can independently explore. A low number of modes suggests a compressed low dimensional neural code and reveals interpretable dynamics, while findings of high dimension may suggest flexible computations. Here, we address the fundamental question of how dimensionality is related to connectivity, in both autonomous and stimulus-driven networks. Working with a simple spiking network model, we derive three main findings. First, the dimensionality of global activity patterns can be strongly, and systematically, regulated by local connectivity structures. Second, the dimensionality is a better indicator than average correlations in determining how constrained neural activity is. Third, stimulus evoked neural activity interacts systematically with neural connectivity patterns, leading to network responses of either greater or lesser dimensionality than the stimulus.

Introduction

As modern neurotechnology enables the simultaneous recording of thousands of neurons, researchers are increasingly confronted with the "curse of dimensionality." While a population of $N$ neurons could theoretically explore an $N$-dimensional space of activity patterns, biological networks often restrict their dynamics to a much smaller subspace. This coordination, quantified as dimensionality, is a critical indicator of a network's computational regime. Low-dimensional activity suggests compressed, robust neural codes, whereas high-dimensional activity may facilitate complex, flexible computations.

Despite its importance, the "missing link" between a circuit's local architecture—the specific "who connects to whom" patterns—and its global dimensionality has remained elusive. Traditional analyses often rely on average pairwise correlations, but these can be misleading. This paper establishes a theoretical bridge between local wiring motifs and global activity trends, demonstrating how recurrent architecture shapes the degrees of freedom available to a neural population.

Methods

To analytically link connectivity to activity, we employed a linearized spiking network model, specifically a linearized inhomogeneous Poisson Generalized Linear Model (GLM) or Hawkes process. This framework provides a balance between analytical tractability and the stochastic nature of biological spiking.

Dimensionality Metric

We quantified dimensionality using the Participation Ratio (PR), derived from the eigenvalues ($\lambda_i$) of the network's covariance matrix: $$PR = \frac{(\sum \lambda_i)^2}{\sum \lambda_i^2}$$ The PR provides a continuous measure of the number of "modes" or degrees of freedom the activity explores.

Motif Expansion and Resumming

The core of our approach involves decomposing the global connectivity matrix into local sub-circuits, or "motifs," involving two or three neurons (e.g., chains, divergent pairs, and reciprocal connections). Using an iterative "resumming" mathematical technique, we derived closed-form expressions that predict global dimensionality based solely on the statistics of these local motifs. This allowed us to test the theory across various topologies, including Erdos-Renyi (ER) networks, Second-Order Network (SONET) models, and Small World or Scale-Free architectures.

Results

Our analysis reveals that highly localized connectivity features—structures that could be governed by local synaptic plasticity rules—exert significant control over a network's global dimensionality. In random ER networks, dimensionality drops precipitously from $N$ toward 1 as average connectivity increases. However, in SONET networks with fixed average connectivity, varying the local motif structure can cause global dimensionality to swing between 20% and 80% of its maximum value.

The "Correlation Trap"

A key finding is that dimensionality is a far more sensitive metric than average pairwise correlation. We observed regimes where dimensionality is extremely low (indicating highly constrained dynamics) even when average correlations are near zero (e.g., 2-3%). This "correlation trap" suggests that simple pairwise measurements can hide the true level of coordination within a network, potentially leading researchers to overestimate the independence of neural activity.

Dimensionality Equalization

When driven by external stimuli, the recurrent network acts as a "dimensionality equalizer." In weakly connected regimes, the network "inflates" low-dimensional inputs into the higher-dimensional space of its internal dynamics. Conversely, in strongly connected regimes, the network acts as a bottleneck, projecting high-dimensional inputs onto a lower-dimensional internal manifold defined by the network's dominant eigenvectors. Reciprocal connections were identified as particularly influential in regulating this balance.

Discussion

While our linearized framework provides powerful insights, it has limitations. Real neurons exhibit non-linearities, such as thresholds and saturation, which may introduce different dimensionality regimes. Furthermore, our current second-order expansion is less precise for balanced excitatory-inhibitory (EI) networks, where inhibitory motifs significantly accelerate the collapse of dimensionality compared to purely excitatory networks.

Future work should extend this framework to higher-order motifs and non-linear dynamics. Applying these principles to "next-generation" connectomics data will help determine if the motif distributions found in biological brains are optimized for the dimensionality requirements of specific biological tasks.

Summary updated

Abstract

Abstract The dimensionality of a network’s collective activity is of increasing interest in neuroscience. This is because dimensionality provides a compact measure of how coordinated network-wide activity is, in terms of the number of modes (or degrees of freedom) that it can independently explore. A low number of modes suggests a compressed low dimensional neural code and reveals interpretable dynamics [1], while findings of high dimension may suggest flexible computations [2, 3]. Here, we address the fundamental question of how dimensionality is related to connectivity, in both autonomous and stimulus-driven networks. Working with a simple spiking network model, we derive three main findings. First, the dimensionality of global activity patterns can be strongly, and systematically, regulated by local connectivity structures. Second, the dimensionality is a better indicator than average correlations in determining how constrained neural activity is. Third, stimulus evoked neural activity interacts systematically with neural connectivity patterns, leading to network responses of either greater or lesser dimensionality than the stimulus. Author summary New recording technologies are producing an amazing explosion of data on neural activity. These data reveal the simultaneous activity of hundreds or even thousands of neurons. In principle, the activity of these neurons could explore a vast space of possible patterns. This is what is meant by high-dimensional activity: the number of degrees of freedom (or “modes”) of multineuron activity is large, perhaps as large as the number of neurons themselves. In practice, estimates of dimensionality differ strongly from case to case, and do so in interesting ways across experiments, species, and brain areas. The outcome is important for much more than just accurately describing neural activity: findings of low dimension have been proposed to allow data compression, denoising, and easily readable neural codes, while findings of high dimension have been proposed as signatures of powerful and general computations. So what is it about a neural circuit that leads to one case or the other? Here, we derive a set of principles that inform how the connectivity of a spiking neural network determines the dimensionality of the activity that it produces. These show that, in some cases, highly localized features of connectivity have strong control over a network’s global dimensionality—an interesting finding in the context of, e.g., learning rules that occur locally. We also show how dimension can be much different than first meets the eye with typical “pairwise” measurements, and how stimuli and intrinsic connectivity interact in shaping the overall dimension of a network’s response.
BibTeX
@inproceedings{recanatesi2019dimensionality,
  author = {Stefano Recanatesi and Gabriel Koch Ocker and Michael A. Buice and Eric Shea‐Brown},
  title = {{Dimensionality in recurrent spiking networks: global trends in activity and local origins in connectivity}},
  booktitle = {PLoS},
  year = {2019},
  doi = {10.1371/journal.pcbi.1006446},
}
Figure