Sensitivity to timing and order in human visual cortex

Jedediah M. Singer, Joseph R. Madsen, William S. Anderson, Gabriel Kreiman

Abstract

Visual recognition takes a small fraction of a second and relies on the cascade of signals along the ventral visual stream. Given the rapid path through multiple processing steps between photoreceptors and higher visual areas, information must progress from stage to stage very quickly. This rapid progression of information suggests that fine temporal details of the neural response may be important to the how the brain encodes visual signals. We investigated how changes in the relative timing of incoming visual stimulation affect the representation of object information by recording intracranial field potentials along the human ventral visual stream while subjects recognized objects whose parts were presented with varying asynchrony. Visual responses along the ventral stream were sensitive to timing differences between parts as small as 17 ms. In particular, there was a strong dependency on the temporal order of stimulus presentation, even at short asynchronies. This sensitivity to the order of stimulus presentation provides evidence that the brain may use differences in relative timing as a means of representing information.

Introduction

Human visual recognition is an incredibly efficient process, occurring in less than 200 milliseconds. This speed is achieved through a hierarchical "cascade" of signals traveling through the ventral visual stream—a series of brain regions dedicated to identifying "what" we see. Because information moves through this system so rapidly (roughly 15ms per stage), a fundamental question arises: does the brain rely on a simple spatial "snapshot" of the world, or do the fine-grained temporal details of the signal carry meaningful information?

This study addresses the problem of spatiotemporal integration: how the brain combines parts of an object that arrive at slightly different times. By manipulating the timing and order of visual inputs, researchers sought to determine if the visual system uses relative timing as a fundamental "code" for representing objects.

Methods

The researchers employed intracranial field potential (IFP) recordings, a powerful technique involving electrodes implanted directly in the brains of five epilepsy patients undergoing clinical monitoring. This method offers superior spatial and temporal resolution compared to non-invasive techniques like fMRI or EEG.

Experimental Design

Subjects performed a visual task involving "chimeric" images—composite stimuli created from two distinct parts (e.g., the top of a torch and the bottom of a lady). These parts were presented with varying Stimulus Onset Asynchrony (SOA), ranging from 0 ms (simultaneous) to 700 ms. The shortest non-zero delay was 17 ms, corresponding to a single refresh cycle on a 60Hz monitor.

Quantitative Modeling

To analyze the neural interactions, three models were tested to predict the brain's response to combined parts:

  1. Simple Linear Model: Assumed the response is a weighted sum of the individual part responses, shifted in time.
  2. General Linear Model (GLM): Allowed weights to vary based on the specific delay and the order of presentation.
  3. Short SOA Model: Predicted responses at longer delays based on the 17ms delay response.

Key Insights

The 17ms Sensitivity Threshold

The human ventral stream is remarkably sensitive to temporal shifts. Even a 17ms delay—which subjects could not consciously distinguish from simultaneous presentation—evoked significantly different neural responses. This suggests the brain "notices" timing differences that the conscious mind ignores.

The Significance of Order

A critical finding was that the neural response depended heavily on the sequence of presentation. Presenting "Part A then Part B" created a different neural signature than "Part B then Part A." This indicates that the visual system is not a passive integrator but is sensitive to the temporal structure of input.

Regional Progression

Timing sensitivity was found as early as the occipital pole (primary visual cortex) but became more pronounced in higher-level areas. In the Inferior Temporal Gyrus (ITG), models accounting for order explained 83% of the neural response, compared to only 31% at the occipital pole.

Results

The General Linear Model, which accounts for both timing and order, explained 62% of the neural responses—a 41% improvement over a simple linear model. Furthermore, the authors developed an Order Tuning Index (OTI) to quantify this sensitivity, finding a strong positive bias across the population. This confirms that the brain's response is more sensitive to a reversal in order than to a change in the length of the delay itself.

Discussion and Limitations

While this study provides compelling evidence for temporal coding in the visual cortex, several limitations remain. The sample size is small (N=5), which is typical for intracranial studies but limits generalizability. Additionally, the study focused on the ventral "what" pathway; the dorsal "where/how" pathway might show even higher sensitivity given its role in motion processing.

Future work should investigate the biological mechanisms behind order sensitivity—perhaps related to "apparent motion" signals—and explore these dynamics using naturalistic stimuli involving continuous motion and eye movements.

Summary updated

Abstract

Visual recognition takes a small fraction of a second and relies on the cascade of signals along the ventral visual stream. Given the rapid path through multiple processing steps between photoreceptors and higher visual areas, information must progress from stage to stage very quickly. This rapid progression of information suggests that fine temporal details of the neural response may be important to the brain's encoding of visual signals. We investigated how changes in the relative timing of incoming visual stimulation affect the representation of object information by recording intracranial field potentials along the human ventral visual stream while subjects recognized objects whose parts were presented with varying asynchrony. Visual responses along the ventral stream were sensitive to timing differences as small as 17 ms between parts. In particular, there was a strong dependency on the temporal order of stimulus presentation, even at short asynchronies. From these observations we infer that the neural representation of complex information in visual cortex can be modulated by rapid dynamics on scales of tens of milliseconds.
BibTeX
@article{singer2014sensitivity,
  author = {Jedediah M. Singer and Joseph R. Madsen and William S. Anderson and Gabriel Kreiman},
  title = {{Sensitivity to timing and order in human visual cortex}},
  journal = {Journal of Neurophysiology},
  year = {2014},
  doi = {10.1152/jn.00556.2014},
}
Figure