The Timing of Vision – How Neural Processing Links to Different Temporal Dynamics

Timothée Masquelier, Larissa Albantakis, Gustavo DecoFrontiers in Psychology 2011 · DOI

Abstract

In this review, we describe our recent attempts to model the neural correlates of visual perception with biologically inspired networks of spiking neurons, emphasizing the dynamical aspects. Experimental evidence suggests distinct processing modes depending on the type of task the visual system is engaged in. A first mode, crucial for object recognition, deals with rapidly extracting the glimpse of a visual scene in the first 100 ms after its presentation. The promptness of this process points to mainly feedforward processing, which relies on latency coding, and may be shaped by spike timing-dependent plasticity (STDP). A second mode can be engaged whenever one needs to perform finer perceptual discrimination through evidence accumulation on the order of 400 ms and above. Here, predominantly local recurrent connections and long neural time-constants enable the integration and build-up of firing rates. A third mode, involving additional top-down attentional signals, shapes visual processing by biasing the competition between different pools of neurons. Finally, oscillatory activity leads to faster reaction times and enhanced information transfers in the model by formatting spike times and encoding information in the spike phases.

Introduction: The Speed-Accuracy Paradox

The primate visual system is a master of contradictions. We can categorize a complex scene in under 100 milliseconds (ms), yet we might take over half a second to decide which way a cluster of noisy dots is moving. Traditional "rate coding" models—which assume neurons convey information by the average number of spikes over time—struggle to explain this. In the first 100ms of vision, a signal passes through roughly ten neuronal layers; there simply isn't enough time for neurons to "average" their firing.

This paper explores how a single biological architecture supports such diverse temporal dynamics by switching between different processing modes—feedforward, recurrent, and oscillatory—depending on the task's demands for speed versus accuracy.

Modeling the Visual Hierarchy

To investigate these dynamics, the authors utilize Spiking Neural Networks (SNNs). Unlike standard artificial neural networks, SNNs model individual electrical pulses (spikes) in time, allowing for the study of temporal coding and biological realism.

Hierarchical architecture of the biologically inspired spiking neural network model, showing layers of simple (S) and complex (C) cells.

Key techniques include:

  • Latency Coding (Rank Order Coding): For ultra-fast tasks, the model assumes the timing of the first spike relative to others carries the information, rather than the total count.
  • Spike-Timing-Dependent Plasticity (STDP): An unsupervised learning rule where synaptic connections are strengthened if an input spike consistently arrives just before the neuron fires.
  • Attractor Networks: For slow decision-making, recurrent loops of excitatory and inhibitory neurons create "attractor states" that allow the network to accumulate evidence over time.

Key Insights: From First Spikes to Oscillations

  • The "First Spike" Advantage: In the first 100ms, the visual system is essentially a "one-spike-per-neuron" machine. STDP naturally acts as a competitive filter, forcing neurons to become selective to the earliest arriving spikes, which tend to be the most salient and informative.
  • NMDA Receptors as "Memory": Deliberate decisions require a high ratio of NMDA to AMPA receptors. The long time-constants of NMDA receptors allow the network to "remember" previous spikes, enabling the build-up of evidence.
  • Oscillations as Traffic Control: Rhythmic neural activity (gamma oscillations) formats spike times into waves. This "Communication Through Coherence" allows higher brain areas to "route" information by aligning their excitability windows with incoming spikes.

Results: Efficiency and Behavioral Realism

The SNN models demonstrate remarkable efficiency and alignment with human psychophysics:

  • Unsupervised Recognition: SNNs equipped with STDP can learn to categorize faces using very few features. As shown in Figure 6, complex objects can be represented and recognized with a minimal number of spikes.
  • Speed-Accuracy Trade-off: Attractor models successfully replicate human behavioral data in motion tasks, showing how increased sensory "noise" leads to slower reaction times and more "changes of mind."
A face image represented by only 100 spikes, demonstrating the efficiency of temporal coding in the model.

Limitations and Future Directions

Current models are largely noise-free and deterministic, whereas real brains are stochastic. Future work must test the robustness of these temporal codes against neuronal variability. Furthermore, most models use "stimulus onset" paradigms, whereas real-world vision is continuous and involves constant eye movements (saccades). Incorporating these feedback loops and moving toward Neuromorphic Hardware—silicon chips that mimic spiking neurons—will be essential for the next generation of energy-efficient visual algorithms.

Summary updated

Abstract

In this review, we describe our recent attempts to model the neural correlates of visual perception with biologically inspired networks of spiking neurons, emphasizing the dynamical aspects. Experimental evidence suggests distinct processing modes depending on the type of task the visual system is engaged in. A first mode, crucial for object recognition, deals with rapidly extracting the glimpse of a visual scene in the first 100 ms after its presentation. The promptness of this process points to mainly feedforward processing, which relies on latency coding, and may be shaped by spike timing-dependent plasticity (STDP). Our simulations confirm the plausibility and efficiency of such a scheme. A second mode can be engaged whenever one needs to perform finer perceptual discrimination through evidence accumulation on the order of 400 ms and above. Here, our simulations, together with theoretical considerations, show how predominantly local recurrent connections and long neural time-constants enable the integration and build-up of firing rates on this timescale. In particular, we review how a non-linear model with attractor states induced by strong recurrent connectivity provides straightforward explanations for several recent experimental observations. A third mode, involving additional top-down attentional signals, is relevant for more complex visual scene processing. In the model, as in the brain, these top-down attentional signals shape visual processing by biasing the competition between different pools of neurons. The winning pools may not only have a higher firing rate, but also more synchronous oscillatory activity. This fourth mode, oscillatory activity, leads to faster reaction times and enhanced information transfers in the model. This has indeed been observed experimentally. Moreover, oscillatory activity can format spike times and encode information in the spike phases with respect to the oscillatory cycle. This phenomenon is referred to as "phase-of-firing coding," and experimental evidence for it is accumulating in the visual system. Simulations show that this code can again be efficiently decoded by STDP. Future work should focus on continuous natural vision, bio-inspired hardware vision systems, and novel experimental paradigms to further distinguish current modeling approaches.
BibTeX
@inproceedings{masquelier2011timing,
  author = {Timothée Masquelier and Larissa Albantakis and Gustavo Deco},
  title = {{The Timing of Vision – How Neural Processing Links to Different Temporal Dynamics}},
  booktitle = {Frontiers in Psychology},
  year = {2011},
  doi = {10.3389/fpsyg.2011.00151},
}
Figure