Abstract
Sensory data about most natural task-relevant variables are entangled with task-irrelevant nuisance variables. The neurons that encode these relevant signals typically constitute a nonlinear population code. Here we present a theoretical framework for quantifying how the brain uses or decodes its nonlinear information. Our theory obeys fundamental mathematical limitations on information content inherited from the sensory periphery, identifying redundant codes when there are many more cortical neurons than primary sensory neurons. The theory predicts that if the brain uses its nonlinear population codes optimally, then more informative patterns should be more correlated with choices. More specifically, the theory predicts a simple, easily computed quantitative relationship between fluctuating neural activity and behavioral choices that reveals the decoding efficiency. We analyze recordings from primary visual cortex of monkeys discriminating the distribution from which oriented stimuli were drawn, and find these data are consistent with the hypothesis of near-optimal nonlinear decoding.
Introduction
In natural environments, sensory information is rarely presented in a clean, isolated format. A neuron in the primary visual cortex (V1) tuned to a specific orientation does not respond solely to that orientation; its activity is also modulated by "nuisance variables" such as spatial phase, lighting conditions, or the stimulus's exact position. These variables entangle the relevant signal, often obscuring the simple linear relationship between a stimulus and a neuron's average firing rate.
To extract meaningful information from such entangled signals, the brain must employ nonlinear decoding—computations that involve complex combinations of neural activities, such as products or higher-order statistics. While established methods exist to measure how well the brain utilizes linear information, a robust framework for quantifying the efficiency of nonlinear decoding has been lacking. This challenge is further complicated by the fact that researchers can typically only record from a minute fraction of the billions of neurons involved in any cognitive task.
The Theoretical Framework
The authors propose a "signature of optimality" based on choice correlations (CC). This framework rests on a core proportionality rule: in an optimal system, the most informative neural patterns must be the ones most strongly correlated with the final behavioral choice. If a specific pattern of activity carries significant information about the task but shows no correlation with the animal's choice, that information is effectively being wasted by the brain's decoder.
The theory extends this proportionality test from simple linear responses to nonlinear functions, such as quadratic statistics. Crucially, it incorporates the concept of "information-limiting correlations." Because large populations of cortical neurons often inherit noise from a much smaller set of sensory inputs, their activity is highly redundant. This redundancy allows the "optimality test" to remain valid even when recording from only a small subset of the total population. To handle tasks where stimulus statistics vary significantly across trials, the researchers developed the Normalized Average Conditional Choice Correlation (NACCC), a statistic that isolates the relevant fluctuations.
Experimental Validation in V1
To test their framework, the researchers analyzed electrophysiological recordings from the V1 of rhesus macaque monkeys. The monkeys performed a variance discrimination task, a 2-alternative forced choice (2AFC) task where they viewed gratings drawn from either a "narrow" or "wide" distribution of orientations. This task is fundamentally nonlinear because the "signal" resides in the variance of the orientation, not the mean.
The results demonstrated that V1 neurons carry substantial nonlinear information about orientation variance, likely due to transformations already performed by "complex cells." Most importantly, the measured nonlinear choice correlations in both monkeys closely aligned with the theoretical predictions for an optimal decoder. For one monkey, the decoding efficiency was calculated at 0.96 ± 0.04, indicating that the brain was utilizing nearly all the nonlinear information available in the recorded V1 population.
Discussion and Limitations
The finding of near-optimal decoding suggests that the brain is remarkably efficient at extracting task-relevant signals from complex, noisy sensory inputs. However, several caveats remain. The current framework primarily assumes a feedforward processing model. While the authors suggest it can be generalized, the brain's extensive feedback loops and recurrent connections add layers of complexity. Feedback from higher brain areas could potentially "mimic" the signature of optimal decoding, making a suboptimal system appear efficient.
Furthermore, applying this test requires researchers to hypothesize which nonlinearities the brain is using (e.g., quadratic vs. cubic). In high-dimensional tasks like object recognition, identifying the relevant complex nonlinearities remains a significant challenge. Future work involving simultaneous recordings across multiple brain areas will be essential to disentangle feedforward decoding from feedback-driven correlations.
Conclusion
This research provides a powerful new tool for neuroscientists to quantify how the brain processes complex information. By linking trial-by-trial neural fluctuations to behavioral choices through a rigorous mathematical framework, it reveals that the primary visual cortex is part of a highly optimized system for nonlinear sensory decoding.
Summary updated