Abstract: A wide range of computations performed by the nervous system involves a type of probabilistic inference known as marginalization. This computation comes up in seemingly unrelated tasks, including causal reasoning, odor recognition, motor control, visual tracking, coordinate transformations, visual search, decision making, and object recognition, to name just a few. The question we address here is: how could neural circuits implement such marginalizations? We show that when spike trains exhibit a particular type of statistics—associated with constant Fano factors and gain-invariant tuning curves, as is often reported in vivo—some of the more common marginalizations can be achieved with networks that implement a quadratic nonlinearity and divisive normalization, the latter being a type of nonlinear lateral inhibition that has been widely reported in neural circuits. Previous studies have implicated divisive normalization in contrast gain control and attentional modulation. Our results raise the possibility that it is involved in yet another, highly critical, computation: near optimal marginalization in a remarkably wide range of tasks.
Introduction
In a world defined by sensory ambiguity, the brain must constantly perform probabilistic inference. A fundamental component of this process is marginalization: the ability to "sum out" or ignore irrelevant "nuisance" variables to focus on the information critical to a task. For instance, to determine an object's location in space, the brain must integrate retinal position with eye position; the eye position is a nuisance variable if the goal is simply to reach for the object. While the brain is known to excel at multisensory integration, the biological mechanism for the complex integration required by marginalization has remained elusive.
This paper proposes that a ubiquitous neural motif—divisive normalization—combined with quadratic nonlinearities, provides a near-optimal solution for marginalization across a vast array of cognitive and sensory domains.
Probabilistic Population Codes (PPCs)
The authors frame their model using Linear Probabilistic Population Codes (PPCs). In this framework, a population of neurons encodes a probability distribution where the log-likelihood of a stimulus is a linear function of the neural activity. A key property of PPCs is that the "gain" (the overall amplitude of the firing rate) is inversely proportional to the variance, or uncertainty, of the encoded information.
When the brain is more certain about a stimulus, the population fires more vigorously. This relationship allows downstream circuits to assess the reliability of incoming signals simply by monitoring the total activity level.
The Quadratic Divisive Normalization (QDN) Model
The core contribution of this research is the Quadratic Divisive Normalization (QDN) model. The researchers propose a multilayer recurrent network where the intermediate layer performs two operations:
- Quadratic Nonlinearity: Input signals are squared or multiplied.
- Divisive Normalization: A neuron's response is divided by the pooled activity of its neighbors, a common form of lateral inhibition.
Mathematically, optimal marginalization requires combining gains in a specific ratio: $g^A = \frac{g^R g^E}{g^R + g^E}$. The authors demonstrate that a circuit implementing QDN naturally produces this exact mathematical form, effectively "automating" Bayesian logic through standard neural architecture.
Benchmarking and Results
The QDN model was tested against several alternatives, including purely linear (L) and purely quadratic (Q) networks, across five simulated benchmarks: linear and nonlinear coordinate transformations, object tracking (Kalman filtering), discrete odor inference, and causal reasoning (the "blicket" task).
In linear coordinate transformations, the QDN network was remarkably efficient, losing only 0.72% of the information present in the input. In contrast, linear models lost approximately 16%. The disparity was even more pronounced in complex tasks like causal reasoning, where standard linear rectified networks lost over 80% of information, while the QDN model lost only 8%.
Furthermore, the model successfully replicated behavioral data from human hand-tracking experiments, suggesting that the QDN motif is not just mathematically sound but biologically relevant to how humans handle motor uncertainty.
Conclusion
Traditionally, divisive normalization has been viewed as a mechanism for gain control or attentional modulation. This work elevates it to a fundamental computational primitive for Bayesian inference. By showing that a single circuit motif can solve problems as diverse as coordinate transformation and causal reasoning, the authors provide a unifying framework for understanding how the brain performs near-optimal probabilistic computations. Future experimental work may confirm these findings by observing how neural saturation points shift in response to the reliability of sensory information.
Summary updated