Optimal population coding by noisy spiking neurons

Gašper Tkačik, Jason Prentice, Vijay Balasubramanian, Elad SchneidmanProceedings of the National Academy of Sciences 2010 · DOI

TL;DR

This paper investigates how noisy spiking neurons in neural populations should interact to maximize the information they represent about stimuli. They found that optimal interactions aren't fixed but smoothly interpolate between error correction, independent encoding, and decorrelation, adapting to noise levels and input statistics. This suggests neural networks are more flexible than previously thought, dynamically adjusting their coding strategies.

What Problem Does This Paper Address?

Neural populations encode information about the world through the spiking activity of their constituent neurons. While it's understood that individual neurons have limitations (noise, limited bandwidth), how the interactions between neurons in a population should be structured to optimize information transmission has been a subject of debate. Some theories suggest neurons should be independent to maximize distinct information representation, while others propose redundancy is crucial for combating noise. This paper tackles the question of what the "optimal" organization of pairwise interactions is in a population of noisy spiking neurons, considering how this optimal structure might vary with the environment (stimulus statistics and noise levels).

Methods

The authors extend the standard linear-nonlinear-Poisson (LNP) model of single neuron firing to include pairwise interactions, creating a stimulus-dependent, pairwise maximum-entropy model. This model is inspired by the "Ising model" from statistical physics, which describes systems with interacting components. They introduce a "neural reliability" parameter ($\beta$) that controls the noise level in individual neurons, ranging from highly reliable (low noise) to very noisy. They then mathematically derive and numerically optimize the parameters of these interactions (specifically, the coupling strengths $J_{ij}$ and intrinsic biases $h_i$) to maximize the mutual information between the stimulus and the population's response. This involves analyzing the trade-off between "output entropy" (the diversity of possible network responses) and "noise entropy" (uncertainty due to noise).

Datasets

Not discussed in the paper. The paper focuses on theoretical modeling and numerical simulations rather than analysis of experimental datasets.

Key Insights and Ideas

  • Continuum of Coding Strategies: The most striking insight is that there isn't a single "best" coding principle. Instead, optimal interactions smoothly transition between error correction (redundancy), independent encoding, and decorrelation based on noise and input statistics.
  • Adaptability is Key: Neural networks are not hardwired with a fixed coding strategy. They can adapt their internal connectivity to optimize information transmission in response to changing environmental conditions (noise and stimulus correlations).
  • Noise Overestimation: Apparent noise measured from individual neurons can significantly overestimate the actual network noise, especially in the presence of error-correcting redundancy.
  • "Learned" Memories: In low-noise regimes, optimal networks develop "attractor patterns" or "basins of attraction" that resemble memories in Hopfield networks, suggesting that the network "learns" and stores information about the stimulus distribution.
  • Efficiency from Subsets: For unreliable neurons, a significant portion of the information can be decoded from a small subset of neurons, highlighting the power of distributed coding.

Key Results

  • Two-Neuron Case: For two neurons, optimal interactions ($J^*$) switch sign depending on neural reliability ($\beta$) and input correlation ($\alpha$). At low $\beta$, $J^*$ matches the input correlation to enhance redundancy and combat noise. At high $\beta$, $J^*$ opposes input correlation to decorrelate outputs.
  • Large Networks: For networks of 10 neurons, similar trends were observed. Optimal networks consistently maintained low noise entropy by adjusting couplings, leading to significant relative increases in information transmission, especially at low neural reliability.
  • Information Gain: Optimal coupling ($J^*$) always improved information transmission compared to uncoupled networks ($J=0$), with gains being larger for strongly correlated inputs.
  • Functional Roles: At low $\beta$, optimal networks implement redundant coding for error correction. At high $\beta$, they perform input decorrelation.

Limitations and Future Work

  • Pairwise Interactions: The model is limited to pairwise interactions. The authors speculate that synergistic coding, observed in some experiments, might require higher-order interactions or might not be optimal for information transmission.
  • Model Simplification: The model uses a simplified representation of spiking neurons (binary outputs) and assumes a stationary stimulus distribution.
  • Experimental Validation: The authors propose that their findings can be tested by comparing predicted optimal connection strengths with experimentally measured ones, and by observing how network connectivity adapts to changes in stimulus statistics. Future work could explore more complex network architectures and stimulus dynamics.

Summary updated

Abstract

In retina and in cortical slice the collective response of spiking neural populations is well described by "maximum-entropy" models in which only pairs of neurons interact. We asked, how should such interactions be organized to maximize the amount of information represented in population responses? To this end, we extended the linear-nonlinear-Poisson model of single neural response to include pairwise interactions, yielding a stimulus-dependent, pairwise maximum-entropy model. We found that as we varied the noise level in single neurons and the distribution of network inputs, the optimal pairwise interactions smoothly interpolated to achieve network functions that are usually regarded as discrete--stimulus decorrelation, error correction, and independent encoding. These functions reflected a trade-off between efficient consumption of finite neural bandwidth and the use of redundancy to mitigate noise. Spontaneous activity in the optimal network reflected stimulus-induced activity patterns, and single-neuron response variability overestimated network noise. Our analysis suggests that rather than having a single coding principle hardwired in their architecture, networks in the brain should adapt their function to changing noise and stimulus correlations.
BibTeX
@inproceedings{tkacik2010optimal,
  author = {Gašper Tkačik and Jason Prentice and Vijay Balasubramanian and Elad Schneidman},
  title = {{Optimal population coding by noisy spiking neurons}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2010},
  doi = {10.1073/pnas.1004906107},
}
Figure