Stimulus content shapes cortical response statistics

Mihály Bányai, Andreea Lazăr, Liane Klein, Johanna Klon-Lipok, Wolf Singer, Gergő OrbánbioRxiv 2017 · DOI

TL;DR

This paper reveals that the way neurons in the visual cortex correlate their activity (spike count correlations or SCCs) isn't just random noise; it's specifically shaped by the complex structure of the visual input. By manipulating the "high-level" features of synthetic images, the researchers demonstrate that these correlations are a direct consequence of hierarchical processing in the brain, where higher-level interpretations influence lower-level neuronal responses.

What Problem Does This Paper Address?

For decades, neuroscience has largely focused on the "mean response" of individual neurons – how strongly they fire on average to a stimulus. However, neurons in the brain don't act in isolation; their activity is correlated. These "noise correlations" or spike count correlations (SCCs) have often been treated as a nuisance, a byproduct of network interactions that might even hinder accurate signal encoding. While some studies have shown that the average level of these correlations can change with stimuli, the fine structure of these correlations and how it relates to the rich, hierarchical nature of visual perception has remained largely unexplored. This paper tackles the question: does the detailed pattern of neuronal correlations carry meaningful, stimulus-specific information, and if so, how does it arise from the brain's processing of complex visual scenes?

Methods

The researchers employed a multi-pronged approach:

  • Electrophysiological Recordings: They recorded from the primary visual cortex (V1) of awake, behaving macaques while presenting various visual stimuli.
  • Natural Images: They used natural images to capture the complexity of real-world visual input.
  • Synthetic Stimuli: To precisely control visual features, they generated synthetic images. This included:
  • Low-level (LL) synthetic stimuli: These mimicked basic visual elements like oriented edges, characteristic of V1 receptive fields, but lacked higher-order structure.
  • High-level (HL) synthetic stimuli: These were designed to incorporate texture-like patterns, representing features processed at higher visual areas (like V2).
  • Contrastive Rate Matching: A novel technique was developed to disentangle the effects of firing rate changes from actual changes in correlation patterns. This is crucial because firing rate and correlations are known to be interdependent.
  • Hierarchical Model: They proposed and analyzed a probabilistic hierarchical model of visual inference to explain their findings.

Datasets

The study used two main types of visual stimuli:

  • Natural Image Patches: These served as a baseline for complex, real-world visual input.
  • Synthetic Image Datasets:
    • LL-synthetic stimuli: Generated from a V1 model, these contained low-level statistical properties but lacked higher-order structure.
  • HL-synthetic stimuli: These incorporated texture-like patterns, mimicking higher-level representations.

No specific benchmark datasets were introduced, but the synthetic stimuli served as controlled experimental conditions.

Key Insights and Ideas

  • Correlation Structure is Stimulus-Specific: The detailed patterns of how neurons' firing rates covary are not random noise but are finely tuned to the specific visual stimulus presented. This goes beyond just changes in the average correlation.
  • High-Level Structure is Key: Removing higher-order statistical structure from synthetic images (creating LL stimuli) significantly reduced the stimulus-specificity of these correlations. Conversely, reintroducing such structure (HL stimuli) restored this specificity.
  • Feedback is the Driver: The findings strongly support the idea that top-down feedback from higher visual areas to V1 is responsible for shaping these stimulus-specific correlations. This feedback allows higher-level interpretations (e.g., "this is a texture") to influence how lower-level features (e.g., edges) are processed and correlated.
  • Hierarchical Inference Explains It All: The observed stimulus-specificity of SCCs is a natural consequence of hierarchical Bayesian inference, where the brain infers the presence of high-level features, which in turn constrains the inference of low-level features.

Key Results

  • Natural Images: Spike count correlation matrices for natural images showed significant stimulus-specificity.
  • LL-Synthetic Stimuli: Synthetic stimuli with only low-level structure exhibited reduced stimulus-specificity in SCCs compared to natural images.
  • HL-Synthetic Stimuli: Synthetic stimuli with reintroduced higher-level structure (texture) restored stimulus-specificity of SCCs, comparable to natural images.
  • Contrastive Rate Matching: This method successfully demonstrated that the observed stimulus-specificity of correlations was independent of changes in neuronal firing rates.
  • Model Predictions: The proposed hierarchical probabilistic model accurately predicted these qualitative changes in stimulus-specificity based on the level of stimulus structure.

Limitations and Future Work

  • Model Simplification: The hierarchical model used V2 as a simplified representation of higher visual areas. Future work could explore more complex hierarchical structures.
  • Generalizability: While the study was conducted in macaques, further research could investigate if these principles apply to other species or sensory modalities.
  • Behavioral Relevance: While the task was designed to engage the animals, a deeper understanding of how these stimulus-specific correlations contribute to actual perceptual behavior and decision-making would be valuable.
  • Mechanistic Details: The study focuses on the functional role of feedback. Future work could delve into the specific neural circuits and synaptic mechanisms underlying these top-down influences.

Summary updated

Abstract

Summary Spike count correlations (SCCs) are ubiquitous in sensory cortices, are characterized by rich structure and arise from structured internal interactions. Yet, most theories of visual perception focus exclusively on the mean responses of individual neurons. Here, we argue that feedback interactions in primary visual cortex (V1) establish the context in which individual neurons process complex stimuli and that changes in visual context give rise to stimulus-dependent SCCs. Measuring V1 population responses to natural scenes in behaving macaques, we show that the fine structure of SCCs is stimulus-specific and variations in response correlations across-stimuli are independent of variations in response means. Moreover, we demonstrate that stimulus-specificity of SCCs in V1 can be directly manipulated by controlling the high-order structure of synthetic stimuli. We propose that stimulus-specificity of SCCs is a natural consequence of hierarchical inference where inferences on the presence of high-level image features modulate inferences on the presence of low-level features.
BibTeX
@misc{banyai2017stimulus,
  author = {Mihály Bányai and Andreea Lazăr and Liane Klein and Johanna Klon-Lipok and Wolf Singer and Gergő Orbán},
  title = {{Stimulus content shapes cortical response statistics}},
  year = {2017},
  doi = {10.1101/238774},
  howpublished = {bioRxiv},
}
Figure