Transformations of sensory information in the brain suggest changing criteria for optimality

Tyler S. Manning, Emma Alexander, Bruce G. Cumming, Gregory C. DeAngelis, Xin Huang, Emily A. Cooper

Introduction

A central tenet of theoretical neuroscience is that the brain is "optimal." However, the definition of optimality is often a moving target. Early in the visual hierarchy, the goal is typically assumed to be Information Maximization (Infomax)—representing as much of the world as possible given limited metabolic resources. As signals move toward behavior-driving areas, the goal logically shifts toward Discrimination Maximization (Discrimax)—focusing precision on the specific stimulus values most relevant for tasks. This paper investigates this transition by examining how the brain encodes binocular disparity across different cortical areas.

The Stimulus: Binocular Disparity

Binocular disparity is the slight difference between the images in our two eyes that allows for depth perception. It is a fundamental sensory signal that must be transformed from raw retinal inputs into useful spatial representations.

Figure 1: Geometry of binocular disparity. A peripheral point projects to different retinal locations relative to the fovea in the left and right eyes, creating the disparity signal used for depth perception.

Methods and Framework

The researchers employed a Fisher Information (FI) framework to bridge the gap between stimulus statistics and neural responses. They analyzed a massive dataset of 1,007 neurons from macaque areas V1, V2, and MT, fitting each neuron’s response with a 1D Gabor function to extract key tuning parameters.

To identify the optimization strategy, they performed a power-law analysis. Theoretical models suggest that if a population is optimal, its Fisher Information ($FI$) should follow a power law relative to the stimulus probability ($p$): $FI \propto p^n$.

  • Infomax predicts $n \approx 2$ (precision follows the most frequent stimuli).
  • Discrimax predicts $n \approx 0.5$ (precision is spread more broadly to support discrimination across a wider range).

Key Results: A Hierarchical Shift

The study found that V1 and V2 are remarkably similar, both leaning heavily toward information preservation with exponents ($n$) of approximately 1.5–1.6. However, as information reaches the middle temporal area (MT), the exponent drops significantly to 0.6–0.9, moving toward the discrimination-optimized regime.

Figure 2: Optimization criteria and error distributions. The L2 norm (top) represents a common penalty function used to define optimality, influencing how errors are distributed across the stimulus space (bottom).

MT does not simply become "noisier." Instead, it reallocates neural resources, moving its peaks of precision away from the most common disparities (near zero) and spreading them out. This suggests the brain is willing to sacrifice absolute precision at the most common depths to gain better discrimination at larger depths. Interestingly, this shift is primarily driven by a change in the preferred disparity (envelope mean) of the neurons. By shifting the population to prefer larger disparities, the brain achieves the transition from Infomax to Discrimax.

Conclusion and Future Directions

These findings suggest that the brain's optimization strategy is not fixed but evolves along the sensory hierarchy. While early stages act as efficient conduits for environmental information, later stages prioritize the fine-grained discrimination needed for behavior. Future work may explore whether these criteria adapt dynamically based on task demands, such as switching between navigation and object manipulation.

Summary updated

Abstract

Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities—the slight differences in the retinal images of the two eyes—we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes.
BibTeX
@article{manning2024transformations,
  author = {Tyler S. Manning and Emma Alexander and Bruce G. Cumming and Gregory C. DeAngelis and Xin Huang and Emily A. Cooper},
  title = {{Transformations of sensory information in the brain suggest changing criteria for optimality}},
  journal = {PLOS Computational Biology},
  year = {2024},
  doi = {10.1371/journal.pcbi.1011783},
}
Figure