Bayesian Inference for Spiking Neuron Models with a Sparsity Prior

Sebastian Gerwinn, Jakob H. Macke, Matthias W. Seeger, Matthias BethgeNeurIPS 2007 · OpenReview

Overview

Predicting how a neuron will spike in response to a stimulus is a fundamental goal in systems neuroscience. While Generalized Linear Models (GLMs) are the standard tool for this task, they struggle with two major hurdles: experimental data is often scarce relative to the massive number of potential parameters, and standard methods (like Maximum Likelihood) fail to provide uncertainty estimates for their predictions.

This paper introduces a fully Bayesian framework for GLMs of spiking neurons. By utilizing a Laplace (sparsity) prior and the Expectation Propagation (EP) algorithm, the authors move beyond simple point estimates to generate a full posterior distribution. This approach provides a principled way to perform automated feature selection and assess the statistical significance of functional connections between neurons.

Methodology: Sparsity and Expectation Propagation

The authors extend the standard GLM by incorporating a Laplace prior over the model weights. Unlike a standard Gaussian prior, the Laplace prior encourages "sparsity"—it aggressively pushes the weights of irrelevant features exactly to zero. This effectively performs automated feature selection, preventing the model from overfitting to noise when training data is limited.

Comparison of Laplace and Gaussian priors on test-set negative log-likelihood across varying training data sizes.

Because the Laplace prior is not differentiable at zero, standard approximation methods fail. To overcome this, the researchers employ Expectation Propagation (EP). EP approximates the complex posterior distribution with a simpler Gaussian distribution by iteratively matching the mean and variance across different factors of the likelihood. The resulting model comprehensively accounts for external stimuli (receptive fields), internal spike history (refractory periods), and couplings (influence from neighboring neurons).

Key Insights and Results

The researchers validated their approach using synthetic data, rabbit retinal ganglion cells (RGCs), and macaque V1 complex cells.

Confidence Intervals for Connectivity Because the method produces a full posterior distribution, it yields confidence intervals (e.g., $\pm 3$ standard deviations) for every weight. This allows researchers to statistically prove whether a "coupling" between two neurons is significant or merely a statistical artifact. In the RGC dataset, the model successfully identified significant functional interactions between specific pairs of retinal cells while correctly dismissing others.

Receptive fields and coupling filters derived from the model. The red lines in the coupling plots (c) represent the confidence intervals, allowing for rigorous significance testing of neuron-to-neuron connections.

Improved Predictive Power In the RGC dataset, the GLM incorporating couplings and spike history significantly outperformed the standard Spike-Triggered Average (STA). The mean correlation coefficient between predicted and actual firing rates rose from 0.1924 (STA) to 0.3699 (GLM with couplings).

Automating Subspace Selection In Spike-Triggered Covariance (STC) analysis, deciding how many filters are relevant is traditionally a manual, heuristic-driven process. This Bayesian approach automates the decision: if a filter isn't needed, the sparsity prior sets its weight to zero. When applied to macaque V1 cells, the model automatically identified 17 significant filters (9 excitatory, 8 inhibitory), closely matching previous studies that relied on manual statistical tests.

Limitations and Future Directions

While highly effective, the approach has limitations. Expectation Propagation is computationally more demanding than simple Maximum A Posteriori (MAP) estimation, which scales poorly as the number of neurons in a population grows. Additionally, while EP was stable in these experiments, it lacks universal convergence guarantees for all non-Gaussian log-concave models. Finally, the current framework assumes stimulus features remain constant between spikes or stimulus updates, which may require adjustment for continuously varying, high-bandwidth inputs.

Summary updated

Abstract

Generalized linear models are the most commonly used tools to describe the stim- ulus selectivity of sensory neurons. Here we present a Bayesian treatment of such models. Using the expectation propagation algorithm, we are able to approximate the full posterior distribution over all weights. In addition, we use a Laplacian prior to favor sparse solutions. Therefore, stimulus features that do not critically influence neural activity will be assigned zero weights and thus be effectively excluded by the model. This feature selection mechanism facilitates both the in- terpretation of the neuron model as well as its predictive abilities. The posterior distribution can be used to obtain confidence intervals which makes it possible to assess the statistical significance of the solution. In neural data analysis, the available amount of experimental measurements is often limited whereas the pa- rameter space is large. In such a situation, both regularization by a sparsity prior and uncertainty estimates for the model parameters are essential. We apply our method to multi-electrode recordings of retinal ganglion cells and use our uncer- tainty estimate to test the statistical significance of functional couplings between neurons. Furthermore we used the sparsity of the Laplace prior to select those filters from a spike-triggered covariance analysis that are most informative about the neural response.
BibTeX
@inproceedings{gerwinn2007bayesian,
  author = {Sebastian Gerwinn and Jakob H. Macke and Matthias W. Seeger and Matthias Bethge},
  title = {{Bayesian Inference for Spiking Neuron Models with a Sparsity Prior}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2007},
  url = {https://openreview.net/forum?id=rJNZ1tW_WS},
}
Figure