Reconstruction of recurrent synaptic connectivity of thousands of neurons from simulated spiking activity

Yury V. Zaytsev, Abigail Morrison, Moritz DegerJ. Comput. Neurosci 2015 · OpenReview

TL;DR

This paper introduces a computationally efficient method to reconstruct the synaptic connectivity of large-scale (thousands of neurons) recurrent neural networks from simulated parallel spike train recordings. By employing a generalized linear model (GLM) with a concave log-likelihood, they achieve accurate connectivity inference, even for complex network structures, overcoming previous computational limitations.

What Problem Does This Paper Address?

Understanding how neural networks function hinges on knowing their connectivity – which neurons connect to which and with what strength. Current experimental techniques for mapping these connections at the cellular level are limited to very small networks and often don't capture the dynamic spiking activity of neurons simultaneously. While computational methods exist, they struggle with the sheer scale of real neural circuits, becoming computationally prohibitive and numerically unstable for networks with thousands of neurons. This paper aims to bridge this gap by providing a scalable and robust method for inferring the detailed synaptic wiring of large, recurrent neural networks from their recorded spiking activity.

Methods

The core of the method is a generalized linear model (GLM) applied to point processes in continuous time. This model describes a neuron's spiking probability based on its own past activity and the activity of other neurons in the network. Key innovations include:

  • Minimal Model for Large Networks: A simplified GLM structure with specific choices for the nonlinearity and interaction kernels, optimized for computational efficiency.
  • Concave Log-Likelihood: The model's log-likelihood function is proven to be concave, guaranteeing that a global optimum can be found using gradient ascent, avoiding local minima issues common in other methods.
  • Efficient Parallelized Optimization: A scheme for numerically optimizing the model parameters that is highly parallelizable, allowing it to scale to thousands of neurons on generic computing clusters. This involves deriving closed-form expressions and recurrence relations for key quantities like membrane potential and gradient calculations.
  • Handling Transmission Delays: The method explicitly incorporates synaptic transmission delays, crucial for realistic network dynamics.
  • Regularization Techniques: The use of $l_1$ regularization is explored to improve reconstruction accuracy, particularly for sparse networks, by imposing a prior belief on the sparsity of the connectivity.

Datasets

The authors validate their method using simulated datasets:

  • Simulated Balanced Random Networks: Networks of 1000 neurons with known ground truth connectivity, used to benchmark the method under ideal conditions and then under progressively less ideal conditions (e.g., model mismatch, distributed parameters).
  • Simulated Synfire Chain: A structured network with an embedded synfire chain, used to demonstrate the method's ability to identify specific network motifs within background activity.

No new experimental datasets were introduced.

Key Insights and Ideas

  • Scalability is Key: The primary breakthrough is enabling the reconstruction of connectivity for networks orders of magnitude larger than previously possible, moving from dozens to thousands of neurons.
  • Concavity is a Game-Changer: Proving the concavity of the log-likelihood function is critical, as it ensures that gradient-based optimization methods will find the true global optimum, not just a local one.
  • Model Simplification for Tractability: The choice of a minimal GLM structure, while still capturing essential dynamics, is what makes the method computationally feasible for large networks.
  • Generative Power of GLMs: The reconstructed GLM not only infers connectivity but also provides a dynamic model that can simulate network activity, allowing for virtual experiments.
  • Clustering Reveals Structure: For complex networks like the synfire chain, clustering the inferred connectivity matrix is essential to reveal hidden structural motifs that are not apparent in a randomized ordering.

Key Results

  • High Accuracy in Ideal Conditions: For a simulated balanced random network of 1000 neurons, synaptic connectivity was recovered with a misclassification error rate (MER) of less than 1% under ideal conditions.
  • Robustness to Imperfect Data: The MER remained low even when faced with model mismatches (e.g., LIF vs. GLM neurons), distributed parameters, and undersampling.
  • Successful Synfire Chain Reconstruction: The method successfully identified a hidden synfire chain embedded in a random network by applying hierarchical clustering to the inferred connectivity.
  • Generative Model Validation: Simulations of the reconstructed synfire chain network showed dynamics virtually identical to the original network when stimulated.
  • Computational Efficiency: The method is designed for efficient parallelization, making it practical for large-scale analyses on modern computing clusters.

Limitations and Future Work

  • Undersampling Effects: While the method performs well, undersampling (not observing all neurons) does lead to a decrease in reconstruction accuracy, as expected. The paper notes that this is a general challenge for connectivity inference.
  • Dale's Law Enforcement: The paper acknowledges that enforcing Dale's law (neurons are either purely excitatory or inhibitory) would turn the problem into a non-convex one, and they opted for an approximate method that yielded minimal gains for their specific datasets.
  • Model Simplifications: The use of exponential kernels for synaptic potentials is a simplification. While effective, more complex kernels could be explored for finer-grained synaptic dynamics.
  • Experimental Validation: The authors highlight the lack of readily available experimental datasets with simultaneous recordings of thousands of neurons and their connectivity, which would be ideal for further validation. They hope for opportunistic validation on future datasets.
  • GPU Acceleration: While a naive GPU implementation showed some speedup, further algorithmic improvements are needed to fully leverage GPU capabilities and remove data transfer bottlenecks.
  • Non-stationarity and External Inputs: The current model assumes stationary parameters. Future work could incorporate external inputs and handle non-stationarity in the data.

Summary updated

Abstract

Dynamics and function of neuronal networks are determined by their synaptic connectivity. Current experimental methods to analyze synaptic network structure on the cellular level, however, cover only small fractions of functional neuronal circuits, typically without a simultaneous record of neuronal spiking activity. Here we present a method for the reconstruction of large recurrent neuronal networks from thousands of parallel spike train recordings. We employ maximum likelihood estimation of a generalized linear model of the spiking activity in continuous time. For this model the point process likelihood is concave, such that a global optimum of the parameters can be obtained by gradient ascent. Previous methods, including those of the same class, did not allow recurrent networks of that order of magnitude to be reconstructed due to prohibitive computational cost and numerical instabilities. We describe a minimal model that is optimized for large networks and an efficient scheme for its parallelized numerical optimization on generic computing clusters. For a simulated balanced random network of 1000 neurons, synaptic connectivity is recovered with a misclassification error rate of less than 1 % under ideal conditions. We show that the error rate remains low in a series of example cases under progressively less ideal conditions. Finally, we successfully reconstruct the connectivity of a hidden synfire chain that is embedded in a random network, which requires clustering of the network connectivity to reveal the synfire groups. Our results demonstrate how synaptic connectivity could potentially be inferred from large-scale parallel spike train recordings.
BibTeX
@article{zaytsev2015reconstruction,
  author = {Yury V. Zaytsev and Abigail Morrison and Moritz Deger},
  title = {{Reconstruction of recurrent synaptic connectivity of thousands of neurons from simulated spiking activity}},
  journal = {J. Comput. Neurosci.},
  year = {2015},
  url = {https://openreview.net/forum?id=N5fv4pjvjf},
}
Figure