Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

Jian K. Liu, Dimokratis Karamanlis, Tim Gollisch

TL;DR

Retinal ganglion cells don't just linearly sum up light intensity; they also respond to the variability of light within their receptive field (spatial contrast). This paper introduces a simple model that incorporates this spatial contrast, significantly improving predictions of how these cells respond to natural images compared to standard linear models. This offers a more accurate and computationally efficient way to understand visual processing in the retina.

What Problem Does This Paper Address?

Understanding how our brains process visual information from the complex, ever-changing world is a fundamental challenge in neuroscience. A key step in this is modeling the behavior of neurons in the retina, specifically retinal ganglion cells (RGCs), which are the output neurons of the retina. While standard models often assume RGCs linearly integrate light intensity across their receptive field, experimental evidence shows that many RGCs exhibit nonlinear spatial integration, especially when presented with stimuli like contrast-reversing gratings. The crucial question is whether these nonlinearities also play a significant role when RGCs process natural, everyday visual scenes, and if so, how to effectively model them without resorting to overly complex or data-intensive approaches.

Methods

The researchers developed a "Spatial Contrast" (SC) model that extends the traditional Linear-Nonlinear (LN) model.

  • Receptive Field Estimation: Standard techniques like spike-triggered average (STA) and singular value decomposition were used to estimate the spatial receptive field of each RGC. This involved presenting spatiotemporal white noise and analyzing the stimulus preceding a spike.
  • LN Model: This model uses the weighted average of light intensity (Imean) within the receptive field as the input to a nonlinear function that predicts the spike count.
  • SC Model: This model adds a second input: local spatial contrast (LSC), calculated as the standard deviation of weighted pixel intensities within the receptive field. The SC model then uses a weighted sum of Imean and LSC as input to the nonlinear function.
  • Parameter Fitting: The parameters for the nonlinear function (and the weight for LSC in the SC model) were optimized using a least-squares criterion on a training set of natural images.
  • Model Evaluation: Performance was quantified by the coefficient of determination (R²) between predicted and measured spike counts on a separate test set of images.
  • Spatial Scale Analysis: To understand the spatial scale of nonlinear integration, the researchers smoothed the input images at different scales before calculating LSC and evaluated how prediction accuracy changed. This helped identify an "optimal" spatial scale for LSC calculation, potentially reflecting the size of underlying subunits (like bipolar cell receptive fields).

Datasets

  • Salamander Retinas: Multielectrode array recordings were made from isolated salamander retinas stimulated with natural images.
  • Mouse Retinal Ganglion Cells: An existing dataset of mouse RGC responses to natural images was analyzed to test the generalizability of the SC model. This dataset included responses to 300 natural images.

Key Insights and Ideas

  • Spatial Contrast Matters: The variability of light intensity (spatial contrast) within an RGC's receptive field, not just the average intensity, significantly influences its firing rate, even for natural images.
  • Simple Model, Big Impact: A surprisingly simple addition—incorporating spatial contrast—to standard models dramatically improves prediction accuracy for RGC responses to natural scenes.
  • Cell-Type Specificity: Larger RGCs, which pool information over wider areas, benefit more from spatial contrast information, suggesting that nonlinearities are more pronounced in these cells.
  • Optimal Spatial Scale: The analysis of spatial smoothing reveals an optimal scale for calculating spatial contrast, which may correspond to the size of underlying neural subunits (e.g., bipolar cell receptive fields) responsible for nonlinear integration.
  • Generalizability: The SC model's effectiveness extends beyond salamanders to mouse RGCs, indicating its broad applicability in understanding retinal processing.

Key Results

  • The SC model improved prediction accuracy (R²) by 10-20% over the classical LN model for salamander RGCs, reaching R² values of up to 0.76 for individual cells.
  • For salamander OFF cells, the SC model showed a significant improvement (average R² of 0.65 vs. 0.53 for LN model, p<10⁻⁹).
  • Large OFF cells in salamanders showed the greatest benefit from spatial contrast information, with prediction improvements of over 2x compared to small cells.
  • In mouse RGCs, the SC model also outperformed the LN model, with average R² values of 0.64 for ON cells and 0.60 for OFF cells, compared to 0.57 and 0.55 for the LN model, respectively.
  • The optimal spatial scale for calculating LSC was typically found to be between 50-150 µm, consistent with the known size of bipolar cell receptive fields.

Limitations and Future Work

  • Phenomenological Model: The SC model is phenomenological; it captures the effect of spatial contrast but doesn't explicitly detail the underlying neural circuitry (e.g., the precise layout and nonlinearities of bipolar cells).
  • Data Requirements: While simpler than full LNLN models, fitting the SC model still requires a reasonable amount of data.
  • Future Directions: The authors suggest extending the SC model to include temporal filtering and feedback mechanisms. They also propose it as a valuable benchmark for more complex models of nonlinear spatial integration, such as LNLN models or those employing deep learning. Further work could explore the optimal spatial scale analysis to infer more about the structure of nonlinear subunits.

Summary updated

Abstract

Abstract A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. We find that response predictions for salamander ganglion cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration in the salamander retina translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields. Author Summary For understanding how sensory systems operate in the natural environment, an important goal is to develop models that capture neuronal responses to natural stimuli. For retinal ganglion cells, which connect the eye to the brain, current standard models often fail to capture responses to natural visual scenes. This shortcoming is at least partly rooted in the fact that ganglion cells may combine visual signals over space in a nonlinear fashion. We here show that a simple model, which not only considers the average light intensity inside a cell’s receptive field but also the variance of light intensity over space, can partly account for these nonlinearities and thereby improve current standard models. This provides an easy-to-obtain benchmark for modeling ganglion cell responses to natural images.
BibTeX
@article{liu2022simple,
  author = {Jian K. Liu and Dimokratis Karamanlis and Tim Gollisch},
  title = {{Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration}},
  journal = {PLOS Computational Biology},
  year = {2022},
  doi = {10.1371/journal.pcbi.1009925},
}
Figure