Optimal prior-dependent neural population codes under shared input noise

Agnieszka Grabska-Barwinska, Jonathan W. Pillow

For decades, neuroscientists have sought to understand the "optimal" shape of neural tuning curves—the mathematical functions describing how neurons fire in response to stimuli, such as the orientation of a line. However, classical theories of neural population coding often fall short, predicting that a small number of simple, independent neurons should be incredibly accurate—far more accurate than actual biological systems.

This discrepancy, known as the efficiency gap, largely stems from oversimplified models. Most classical literature relies on Fisher Information to measure coding quality. While mathematically convenient, Fisher Information is a local metric that often fails when spike counts are low or when the brain has strong "prior" expectations about the world. Furthermore, these models typically ignore a crucial biological reality: noise often corrupts a stimulus before it is even encoded by the brain.

This paper bridges the gap by introducing a mathematically tractable model that incorporates input noise—noise that cannot be averaged away by simply adding more neurons—and stimulus-dependent correlations.

The neural encoding model incorporating input noise and a Bayesian decoding framework.

The Encoding Model and Biological Constraints

The authors analyze a population of neurons with Gaussian-shaped tuning curves designed to "tile" the stimulus space, ensuring their combined baseline activity remains constant across the stimulus range.

To understand optimal tuning, the researchers compared two distinct biological constraints:

  • Space Constraint: The system has a fixed number of neurons. If tuning curves are made wider, the total number of spikes in the population increases because more neurons "see" and respond to the stimulus.
  • Energy Constraint: The system has a fixed "budget" of spikes. If tuning curves are widened, their peak firing rates (height) must be lowered to keep the total spike count constant.

Crucially, the authors move beyond Fisher Information, deriving exact Bayesian expressions for Mean Squared Error (MSE) and Mutual Information (MI) to evaluate how well the neural response captures the stimulus.

Key Insights: Breaking the Fisher Information Illusion

By utilizing analytical derivations and Monte Carlo simulations, the study reveals several paradigm-shifting insights:

  • Input Noise is a "Hard Ceiling": Unlike "output noise" (randomness in a neuron's firing mechanism), input noise creates irreducible uncertainty. No matter how many neurons are added to the population, the system can never be more accurate than the noisy signal that initially entered it.
  • The "Space vs. Energy" Divide: Under a space constraint, there is an optimal, non-zero tuning width. If curves are too narrow, the brain might miss the stimulus entirely; if too wide, the signal blurs. Conversely, under an energy constraint, the math suggests tuning curves should be as narrow as possible (down to the limit of the tiling assumption).
  • Fisher Information Can Lie: The authors demonstrate that Fisher Information often misleadingly suggests that narrower tuning is always better. However, their Bayesian analysis reveals that when spike counts are low or priors are narrow, wider tuning is actually superior.
  • Signal-Dependent Correlations: The input noise model naturally produces correlations between neurons that change depending on the stimulus, mimicking the complex "noise correlations" observed in real visual cortex data.

Results: Optimal Width and Prior Expectations

The authors provide new, tight analytical bounds for Mutual Information and Mean Squared Error that are significantly more accurate than classical Cramér-Rao bounds for low spike counts.

A standout result is the relationship between optimal tuning width and the brain's prior expectations. As the brain's "prior" (its expectation of the stimulus) becomes wider or more uncertain, the optimal neural tuning width must also increase to compensate.

Effects of prior standard deviation and time window on Mutual Information under space and energy constraints.

Furthermore, under input noise, the benefit of increasing tuning width eventually saturates. Once the tuning is wide enough to capture the spread of the input noise, making it wider provides diminishing returns for Mutual Information.

Limitations and Future Directions

While mathematically elegant, the model relies on Gaussian assumptions for both tuning curves and stimulus priors. Real-world stimuli, such as natural images, feature much more complex distributions. Additionally, the paper focuses on a simple 1D scalar stimulus, leaving the extension to high-dimensional sensory inputs as a future challenge. Finally, the model is "static" and does not account for temporal dynamics or how the brain might adapt its tuning curves during an active task.

Despite these limitations, this "input noise" framework provides a highly realistic testbed for re-examining experimental data, offering a more robust mathematical foundation for understanding why the brain is wired the way it is.

Summary updated

Abstract

The brain uses population codes to form distributed, noise-tolerant representations of sensory and motor variables. Recent work has examined the theoretical optimality of such codes in order to gain insight into the principles governing population codes found in the brain. However, the majority of the population coding literature considers either conditionally independent neurons or neurons with noise governed by a stimulus-independent covariance matrix. Here we analyze population coding under a simple alternative model in which latent input noise" corrupts the stimulus before it is encoded by the population. This provides a convenient and tractable description for irreducible uncertainty that cannot be overcome by adding neurons, and induces stimulus-dependent correlations that mimic certain aspects of the correlations observed in real populations. We examine prior-dependent, Bayesian optimal coding in such populations using exact analyses of cases in which the posterior is approximately Gaussian. These analyses extend previous results on independent Poisson population codes and yield an analytic expression for squared loss and a tight upper bound for mutual information. We show that, for homogeneous populations that tile the input domain, optimal tuning curve width depends on the prior, the loss function, the resource constraint, and the amount of input noise. This framework provides a practical testbed for examining issues of optimality, noise, correlation, and coding fidelity in realistic neural populations."
BibTeX
@inproceedings{grabskabarwinska2014optimal,
  author = {Agnieszka Grabska-Barwinska and Jonathan W. Pillow},
  title = {{Optimal prior-dependent neural population codes under shared input noise}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2014},
}
Figure