The Equivalence of Information-Theoretic and Likelihood-Based Methods for Neural Dimensionality Reduction

Ross S. Williamson, Maneesh Sahani, Jonathan W. PillowPLOS Computational Biology 2015 · DOI · OpenReview

In systems neuroscience, identifying the specific stimulus features that trigger neuronal activity is a fundamental challenge. Because natural stimuli are high-dimensional, researchers often employ dimensionality reduction techniques to find a low-dimensional subspace of features relevant to a neuron's response. A prominent tool for this task has been Maximally Informative Dimensions (MID), an information-theoretic method favored for its perceived lack of assumptions regarding a neuron's firing statistics.

This research, published in PLOS Computational Biology, provides a rigorous mathematical bridge between information theory and statistical modeling. The authors demonstrate that MID is not "assumption-free" but is instead mathematically equivalent to a specific class of statistical models.

The Equivalence Proof: MID as a Poisson Model

The core contribution of this work is the proof that the "single-spike information" optimized by MID is exactly equal to the log-likelihood of a Linear-Nonlinear-Poisson (LNP) model. This finding implies that using MID implicitly assumes a Poisson distribution for neuronal spiking. By establishing this link, the authors transform an information-theoretic problem into a maximum-likelihood estimation problem, opening the door to more robust and scalable statistical techniques.

Generalizing to Non-Poisson Neurons

Recognizing MID's implicit Poisson assumption allows for its generalization to neurons with different firing characteristics. The authors introduce two new models to handle non-Poisson statistics:

  • Linear-Nonlinear-Bernoulli (LNB): Designed for neurons that fire at most one spike per time bin, effectively capturing refractory periods.
  • Linear-Nonlinear-Count (LNC): A flexible framework for arbitrary spike count distributions.

A key insight from this generalization is that for non-Poisson neurons, the absence of a spike (silence) carries significant information. Traditional MID, which focuses only on stimuli that trigger spikes, ignores this data. The authors show that neglecting silences leads to biased estimates of a neuron's preferred features, particularly in binary spike trains where MID can lose at least half of the available information.

Overcoming the Curse of Dimensionality

Traditional MID often struggles with the "curse of dimensionality," typically failing to identify more than two or three stimulus features due to the exponential growth of parameters in histogram-based methods. To address this, the authors propose using Cylindrical Basis Functions (CBFs). These parametric functions act as localized "bumps" in low-dimensional projections while remaining constant in other dimensions. This approach allows the model to estimate a much larger number of stimulus features without an explosion in parameter count, providing a smoother and more scalable alternative to histograms.

Results and Validation

The proposed methods were validated using both synthetic simulations and real-world data from the primate visual cortex (V1).

Synthetic Simulations

In tests with Bernoulli and deterministic neurons, traditional MID exhibited significant bias—sometimes erring by as much as 90 degrees in its estimate of the preferred stimulus axis. In contrast, the likelihood-based LNB and LNC models accurately recovered the true stimulus features by accounting for the information contained in periods of silence.

Primate V1 Data

When applied to extracellular recordings from macaque V1 neurons stimulated with flickering bars, the CBF-based LNP model outperformed common covariance-based methods like iSTAC. The new method successfully recovered up to eight informative filters, revealing that excitatory filters are generally more informative than suppressive ones—a nuance often obscured by previous techniques.

Conclusion and Future Directions

By unifying information-theoretic and likelihood-based approaches, this research provides a more principled and scalable framework for neural dimensionality reduction. While current models assume temporal independence between spikes, future work could integrate these insights into Generalized Linear Models (GLMs) to account for spike-history effects. This synthesis not only clarifies the underlying assumptions of popular tools like MID but also provides the computational machinery needed to explore the high-dimensional feature spaces of the brain more deeply.

Summary updated

Abstract

Author Summary A popular approach to the neural coding problem is to identify a low-dimensional linear projection of the stimulus space that preserves the aspects of the stimulus that affect a neuron’s probability of spiking. Previous work has focused on both information-theoretic and likelihood-based estimators for finding such projections. Here, we show that these two approaches are in fact equivalent. We show that maximally informative dimensions (MID), a popular information-theoretic method for dimensionality reduction, is identical to the maximum-likelihood estimator for a particular linear-nonlinear encoding model with Poisson spiking. One implication of this equivalence is that MID may not find the information-theoretically optimal stimulus projection when spiking is non-Poisson, which we illustrate with a few simple examples. Using these insights, we propose novel dimensionality-reduction methods that incorporate non-Poisson spiking, and suggest new parametrizations that allow for tractable estimation of high-dimensional subspaces.
BibTeX
@article{williamson2015equivalence,
  author = {Ross S. Williamson and Maneesh Sahani and Jonathan W. Pillow},
  title = {{The Equivalence of Information-Theoretic and Likelihood-Based Methods for Neural Dimensionality Reduction}},
  journal = {PLOS Computational Biology},
  year = {2015},
  doi = {10.1371/journal.pcbi.1004141},
}
Figure