Characterizing the nonlinear structure of shared variability in cortical\n neuron populations using latent variable models

Matthew R Whiteway, Karolina Socha, Vincent Bonin, Daniel A. ButtsarXiv 2019 · DOI · arXiv

Abstract

Sensory neurons often have variable responses to repeated presentations of the same stimulus, which can significantly degrade the stimulus information contained in those responses. This information can in principle be preserved if variability is shared across many neurons, but depends on the structure of the shared variability and its relationship to sensory encoding at the population level. The structure of this shared variability in neural activity can be characterized by latent variable models, although they have thus far typically been used under restrictive mathematical assumptions. Here we introduce two nonlinear latent variable models for analyzing large-scale neural recordings. We first present a general nonlinear latent variable model that is agnostic to the stimulus tuning properties of the individual neurons, and is hence well suited for exploring neural populations whose tuning properties are not well characterized. This motivates a second class of model, the Generalized Affine Model, which simultaneously determines each neuron's stimulus selectivity and a set of latent variables that modulate these stimulus-driven responses both additively and multiplicatively. While these approaches can detect very general nonlinear relationships in shared neural variability, we find that neural activity recorded in anesthetized primary visual cortex (V1) is best described by a single additive and single multiplicative latent variable, i.e. an 'affine model'. In contrast, application of the same models to recordings in awake macaque prefrontal cortex discover more general nonlinearities to compactly describe the population response variability. These results thus demonstrate how nonlinear latent variable models can be used to describe population response variability, and suggest that a range of methods is necessary to study different brain regions under different experimental conditions.

Introduction

Neurons in the brain are notoriously "noisy," responding differently to the exact same stimulus across multiple trials. Crucially, this variability is often shared across many neurons (known as noise correlations), which can either hinder or help the brain's ability to encode information. To study this, researchers use latent variable models to identify hidden "factors" that explain the activity of hundreds of neurons.

However, traditional methods like Principal Component Analysis (PCA) or Factor Analysis (FA) assume these relationships are linear. Given the inherent nonlinearities of neural circuits—such as thresholding and saturation—linear models often fail to capture the true structure of population activity. This paper introduces nonlinear models that are flexible enough to fit complex data while remaining structured enough to be interpretable.

Methods

The authors propose two frameworks based on autoencoder-style neural networks:

  1. Stacked Rectified Latent Variable Model (SRLVM): A "stimulus-agnostic" model. It uses deep neural networks to encode high-dimensional neural activity into a low-dimensional latent space and then decode it back into predicted firing rates. Because it does not explicitly incorporate stimulus information, it must capture both stimulus-driven tuning and internal variability within the latents.
  2. Generalized Affine Model (GAM): A more structured approach that explicitly incorporates stimulus tuning. It models a neuron's response as a combination of its stimulus-driven response modulated by shared latent variables. Specifically, it allows for multiple multiplicative (gain-like) and additive (offset-like) latent factors.
The SRLVM architecture (A) and its performance on simulated data (B) compared to a linear model (RLVM).

Both models are fit by maximizing log-likelihood under a Gaussian noise assumption (using square-rooted spike counts) and utilize $L_2$ regularization to prevent overfitting. The models were validated on recordings from the primary visual cortex (V1) of anesthetized macaques and the prefrontal cortex (PFC) of awake macaques.

Key Results and Insights

The "Affine" Nature of V1

One of the most striking findings is that in anesthetized V1, a very simple model—one additive and one multiplicative latent variable—captures almost as much variance as the most complex deep neural network. This suggests that V1 variability under anesthesia is dominated by simple, global fluctuations in gain and offset. The Quality Index (QI) difference between the simple affine model and the complex SRLVM was negligible in this region.

PFC Complexity

In contrast, the awake PFC is much more "high-dimensional." The simple affine model fails significantly here. To match the performance of the general SRLVM, the GAM requires many more latent variables (up to 9 additive and 9 multiplicative factors). The SRLVM's ability to outperform simpler models in certain regimes suggests that there are nonlinear interactions between latents and stimuli that go beyond simple scaling or shifting.

Quality Index vs. number of latent variables for PFC data (Monkey 1), showing the performance gap between the nonlinear SRLVM and the linear RLVM.

Discussion

These results demonstrate how nonlinear latent variable models can be used to describe population response variability. While V1 under anesthesia appears relatively simple, the awake PFC exhibits complex nonlinear interactions. It remains unclear if this difference is due to the brain region itself or the cognitive demands of the awake state. Future work could extend these models to account for within-trial temporal dynamics or utilize fully probabilistic frameworks like Variational Autoencoders for better uncertainty quantification.

Summary updated

Abstract

Sensory neurons often have variable responses to repeated presentations of\nthe same stimulus, which can significantly degrade the stimulus information\ncontained in those responses. This information can in principle be preserved if\nvariability is shared across many neurons, but depends on the structure of the\nshared variability and its relationship to sensory encoding at the population\nlevel. The structure of this shared variability in neural activity can be\ncharacterized by latent variable models, although they have thus far typically\nbeen used under restrictive mathematical assumptions. Here we introduce two\nnonlinear latent variable models for analyzing large-scale neural recordings.\nWe first present a general nonlinear latent variable model that is agnostic to\nthe stimulus tuning properties of the individual neurons, and is hence well\nsuited for exploring neural populations whose tuning properties are not well\ncharacterized. This motivates a second class of model, the Generalized Affine\nModel, which simultaneously determines each neuron's stimulus selectivity and a\nset of latent variables that modulate these stimulus-driven responses both\nadditively and multiplicatively. While these approaches can detect very general\nnonlinear relationships in shared neural variability, we find that neural\nactivity recorded in anesthetized primary visual cortex (V1) is best described\nby a single additive and single multiplicative latent variable, i.e. an `affine\nmodel'. In contrast, application of the same models to recordings in awake\nmacaque prefrontal cortex discover more general nonlinearities to compactly\ndescribe the population response variability. These results thus demonstrate\nhow nonlinear latent variable models can be used to describe population\nresponse variability, and suggest that a range of methods is necessary to study\ndifferent brain regions under different experimental conditions.\n
BibTeX
@misc{whiteway2019characterizing,
  author = {Matthew R Whiteway and Karolina Socha and Vincent Bonin and Daniel A. Butts},
  title = {{Characterizing the nonlinear structure of shared variability in cortical\textbackslash\{\}n neuron populations using latent variable models}},
  year = {2019},
  eprint = {1904.10441},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1904.10441},
}
Figure