A new inference approach for training shallow and deep generalized linear models of noisy interacting neurons

Gabriel Mahuas, Giulio Isacchini, Olivier Marre, Ulisse Ferrari, Thierry Mora

Introduction

In computational neuroscience, Generalized Linear Models (GLMs) are a foundational tool for predicting how neurons fire in response to stimuli. They are widely used because they are relatively easy to fit and can account for both the external stimulus and the "couplings"—how one neuron’s firing influences its neighbors.

However, researchers have identified a major Achilles' heel in standard GLMs: they struggle with complex, highly correlated stimuli like natural videos. To address this, a novel "two-step" inference strategy has been developed that separates stimulus-driven activity from neuron-to-neuron interactions. This approach prevents models from "hallucinating" network connections based on stimulus patterns, leading to better generalization and eliminating the instabilities common in traditional neural models.

The Challenge with Standard GLMs

When a stimulus has strong spatial or temporal patterns—such as a bar moving across a screen—a standard GLM often gets confused. It might erroneously attribute the synchronized firing of two neurons to a direct biological connection between them, when in reality, both neurons are simply responding to the same moving bar.

This "mixing trap" leads to three significant problems:

  1. Lack of Generalization: A model trained on one type of visual environment (e.g., moving bars) fails when tested on another (e.g., random checkerboards).
  2. Instability: Overestimated couplings can cause the model to enter a "self-excitation" loop, where neurons trigger each other in an unrealistic, runaway explosion of activity (often called a "blowup").
  3. The CNN Gap: While Deep Convolutional Neural Networks (CNNs) excel at predicting mean firing rates from complex stimuli, they typically lack the coupling mechanisms of GLMs needed to explain shared noise and fine-scale neural correlations.

A Two-Step Inference Strategy

To disentangle stimulus effects from internal network dynamics, the authors propose a two-step inference strategy:

  1. Coupling Inference: Instead of training the entire model simultaneously, this step isolates "noise correlations" (internal network effects) from "stimulus correlations" by analyzing responses to repeated stimulus trials. By observing how neurons vary across identical repetitions, the model uses auxiliary variables to perfectly match the mean response (PSTH). This forces the coupling parameters to focus solely on true neuron-to-neuron interactions, making them stimulus-independent.
  2. Filter Inference: In a separate step, the stimulus-processing component (the "filter") is trained. This can be a standard linear filter or a more complex Deep CNN.
  3. Integration: Finally, the two components are combined using a mathematical correction. Because adding couplings alters the predicted firing rate, the mean contribution of the coupling terms is subtracted to ensure the final model's firing rate predictions remain accurate.

Key Results and Performance

The study validated this approach using multi-electrode array recordings from rat retinal ganglion cells, testing the models against both low-correlation stimuli (white noise checkerboards) and highly correlated stimuli (moving bars and naturalistic movies).

Performance comparison of standard and two-step GLMs across different stimuli, highlighting the elimination of runaway excitation (blowups) in panel H.

The two-step method demonstrated significant improvements over traditional approaches:

  • Improved Accuracy: On the challenging "moving bars" dataset, the two-step approach achieved a Coefficient of Determination (CoD) for noise correlations of 0.91, a massive improvement over the 0.55 achieved by the standard one-step GLM.
  • Eliminating Blowups: Standard GLMs suffered from self-excitation transients 10% of the time (affecting 36% of repetitions). The two-step model completely eliminated this issue, exhibiting 0% self-excitation. The instabilities in older models were not a fundamental architectural flaw, but a symptom of overestimated coupling strengths.
  • Robust Generalization: Because the isolated couplings reflect true biological interactions rather than stimulus artifacts, a model trained on moving bars can accurately predict activity during a checkerboard stimulus, and vice versa. Standard models become unstable under these cross-testing conditions.
  • Deep GLM Integration: By plugging a Deep CNN into the framework as the stimulus processor, the researchers improved the prediction of individual neuron firing rates (Pearson’s $\rho = 0.87$ vs $0.82$) while maintaining high-fidelity correlation predictions.

Limitations and Future Directions

The primary requirement of this new method is the need for repeated stimulus data to perform the initial coupling inference. While the authors demonstrated that only 165 seconds of repeated data was necessary for the retina, obtaining such data can be challenging in certain experimental setups, such as in vivo recordings in higher brain areas.

Future work may explore using spontaneous activity (e.g., firing in the dark) to infer couplings, which would increase the method's flexibility. Additionally, applying this framework to even more complex natural scenes will help determine if the decoupling remains robust when stimulus statistics are highly non-Gaussian.

Summary updated

Abstract

Generalized linear models are one of the most efficient paradigms for predicting the correlated stochastic activity of neuronal networks in response to external stimuli, with applications in many brain areas. However, when dealing with complex stimuli, the inferred coupling parameters often do not generalize across different stimulus statistics, leading to degraded performance and blowup instabilities. Here, we develop a two-step inference strategy that allows us to train robust generalized linear models of interacting neurons, by explicitly separating the effects of correlations in the stimulus from network interactions in each training step. Applying this approach to the responses of retinal ganglion cells to complex visual stimuli, we show that, compared to classical methods, the models trained in this way exhibit improved performance, are more stable, yield robust interaction networks, and generalize well across complex visual statistics. The method can be extended to deep convolutional neural networks, leading to models with high predictive accuracy for both the neuron firing rates and their correlations.
BibTeX
@inproceedings{mahuas2020new,
  author = {Gabriel Mahuas and Giulio Isacchini and Olivier Marre and Ulisse Ferrari and Thierry Mora},
  title = {{A new inference approach for training shallow and deep generalized linear models of noisy interacting neurons}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
}
Figure