Multiple Object Response Normalization in Monkey Inferotemporal Cortex

Davide Zoccolan, David Cox, James J. DiCarloJournal of Neuroscience 2005 · DOI

Abstract

The highest stages of the visual ventral pathway are commonly assumed to provide robust representation of object identity by disregarding confounding factors such as object position, size, illumination, and the presence of other objects (clutter). However, whereas neuronal responses in monkey inferotemporal cortex (IT) can show robust tolerance to position and size changes, previous work shows that responses to preferred objects are usually reduced by the presence of nonpreferred objects. More broadly, we do not yet understand multiple object representation in IT. In this study, we systematically examined IT responses to pairs and triplets of objects in three passively viewing monkeys across a broad range of object effectiveness. We found that, at least under these limited clutter conditions, a large fraction of the response of each IT neuron to multiple objects is reliably predicted as the average of its responses to the constituent objects in isolation. That is, multiple object responses depend primarily on the relative effectiveness of the constituent objects, regardless of object identity. This average effect becomes virtually perfect when populations of IT neurons are pooled. Furthermore, the average effect cannot simply be explained by attentional shifts but behaves as a primarily feedforward response property. Together, our observations are most consistent with mechanistic models in which IT neuronal outputs are normalized by summed synaptic drive into IT or spiking activity within IT and suggest that normalization mechanisms previously revealed at earlier visual areas are operating throughout the ventral visual stream.

Introduction

The primate ventral visual stream is a hierarchy of processing stages culminating in the Inferotemporal (IT) cortex, where neurons represent complex object identities. While it is well-established that IT neurons exhibit robust tolerance to changes in object position and size, their response to visual clutter—the presence of multiple objects in the same receptive field—remains less understood. Traditional models suggest either a "MAX" response (clutter invariance) or top-down attentional filtering. This study investigates whether a systematic integration rule governs IT responses to multiple objects.

Experimental Design and Methods

Extracellular recordings were obtained from 104 single IT neurons in three rhesus monkeys across two experimental paradigms. Experiment 1 utilized simple geometric shapes (e.g., stars, crosses) in isolation, pairs, and triplets. Experiment 2 employed parametrically morphed objects, including 3D cars, human heads, and NURBS silhouettes, to test the influence of shape similarity on response integration.

Overview of experimental stimuli and presentation paradigms. (A) Stimuli for Exp 1 and 2. (B-D) Single, pair, and triplet conditions. (E) Rapid serial visual presentation (RSVP) timeline.

Monkeys performed a passive viewing task using a rapid serial visual presentation (RSVP) protocol. Stimuli were presented for 100ms, a duration intended to capture core feedforward responses while minimizing the influence of top-down attentional shifts. Neuronal responses were compared against two primary models: an Average Model ($R_{AB} \approx (R_A + R_B) / 2$) and a MAX Model ($R_{AB} \approx \max(R_A, R_B)$).

Key Insights: The Normalization Rule

The study's primary finding is that IT neurons respond to multiple objects by averaging their responses to the constituent objects. This suggests that divisive normalization, a canonical computation previously identified in earlier visual areas like V1 and V4, is also a fundamental property of the highest object-recognition stages.

Notably, this averaging rule is independent of object identity or shape similarity. The response depends primarily on the relative effectiveness of the constituent objects. Even an "ineffective" distractor—one that elicits no response in isolation—can significantly suppress the neuron's response to a preferred object when presented together.

Quantitative Results

Quantitative analysis strongly supported the averaging model. Scatter plots of pair responses versus the sum of individual responses yielded slopes of 0.55 and 0.58 for Experiments 1 and 2, respectively, closely matching the theoretical 0.5 predicted by averaging. For triplets, the slope was 0.37, near the predicted 0.33 (1/3). The average model accounted for approximately 63% of the explainable variance.

Furthermore, quantile-quantile (Q-Q) plot analysis of spike distributions revealed unimodal distributions, ruling out trial-by-trial attentional switching as the underlying mechanism. This confirms that the observed averaging is a true neural integration process rather than an attentional artifact.

Discussion and Future Directions

While these results provide strong evidence for normalization in IT, the study's scope was limited to 2-3 objects in a passive viewing context. Future work is needed to determine if this rule holds in high-density clutter or under active task demands where top-down attention might modulate these feedforward properties. Additionally, the exact mechanistic origin of this normalization—whether it is computed within IT or inherited from earlier stages—remains to be determined.

Reference: Zoccolan, D., Cox, D. D., & DiCarlo, J. J. (2005). Multiple Object Response Normalization in Monkey Inferotemporal Cortex. Journal of Neuroscience, 25(36), 8150-8164.

Summary updated

Abstract

The highest stages of the visual ventral pathway are commonly assumed to provide robust representation of object identity by disregarding confounding factors such as object position, size, illumination, and the presence of other objects (clutter). However, whereas neuronal responses in monkey inferotemporal cortex (IT) can show robust tolerance to position and size changes, previous work shows that responses to preferred objects are usually reduced by the presence of nonpreferred objects. More broadly, we do not yet understand multiple object representation in IT. In this study, we systematically examined IT responses to pairs and triplets of objects in three passively viewing monkeys across a broad range of object effectiveness. We found that, at least under these limited clutter conditions, a large fraction of the response of each IT neuron to multiple objects is reliably predicted as the average of its responses to the constituent objects in isolation. That is, multiple object responses depend primarily on the relative effectiveness of the constituent objects, regardless of object identity. This average effect becomes virtually perfect when populations of IT neurons are pooled. Furthermore, the average effect cannot simply be explained by attentional shifts but behaves as a primarily feedforward response property. Together, our observations are most consistent with mechanistic models in which IT neuronal outputs are normalized by summed synaptic drive into IT or spiking activity within IT and suggest that normalization mechanisms previously revealed at earlier visual areas are operating throughout the ventral visual stream.
BibTeX
@article{zoccolan2005multiple,
  author = {Davide Zoccolan and David Cox and James J. DiCarlo},
  title = {{Multiple Object Response Normalization in Monkey Inferotemporal Cortex}},
  journal = {Journal of Neuroscience},
  year = {2005},
  doi = {10.1523/jneurosci.2058-05.2005},
}
Figure