Visual focus with spiking neurons

Sylvain Chevallier, Philippe TarrouxESANN 2008 · OpenReview

Abstract

This paper presents a biologically inspired visual attention model using Leaky Integrate-and-Fire (LIF) spiking neurons. Moving away from traditional rate-based Neural Field Theory, the proposed two-layer network—comprising an Input Map and a Focus Map—demonstrates robust tracking of moving targets amidst significant background noise and distractors. By leveraging spike timing and lateral inhibition, the model achieves high computational efficiency and superior filtering capabilities, maintaining stable performance where traditional input-level processing fails.

Introduction

In both computer vision and computational neuroscience, "attentional focusing" is the critical ability to lock onto a specific stimulus while ignoring irrelevant information. Many existing models utilize Neural Field Theory, which relies on "rate coding"—treating neural activity as a continuous average of firing frequency. However, biological brains communicate via discrete electrical pulses called spikes. This research investigates whether a spiking neural network (SNN) can replicate the success of rate-based models while potentially improving robustness through precise spike timing. The goal is to implement an "early" visual attention mechanism that filters noise before higher-level processing occurs.

Methods

The architecture consists of two 2D neural maps utilizing Leaky Integrate-and-Fire (LIF) neurons. In this model, a neuron's membrane potential builds as it receives signals, decays over time, and resets upon reaching a firing threshold.

  1. Input Map (IM): These neurons act as integrators, converting pixel luminance from an input image into spike trains. Brighter pixels result in higher firing frequencies.
  2. Focus Map (FM): These neurons function as synchrony detectors. They receive Gaussian-weighted input from the IM and utilize lateral inhibition via a "Mexican Hat" (Difference of Gaussians) connectivity pattern. This mechanism allows a firing neuron to excite immediate neighbors while inhibiting those further away, effectively narrowing the network's focus to a single point of activity.
Architecture of the two-layer spiking neural network showing the Input Map and Focus Map.

Experimental Setup

The model was tested using a synthetic 30x30 pixel image sequence featuring a Gaussian "patch" moving in a circular path. To test robustness, the authors introduced two primary challenges:

  • Noise: Independent zero-mean Gaussian noise added to the background.
  • Distractors: Visually identical stimuli that appear and disappear randomly, lacking the spatio-temporal continuity of the primary target.

Results and Discussion

The Focus Map (FM) demonstrated remarkable stability compared to the raw Input Map (IM). While the IM's error increased linearly with noise and the number of distractors, the FM remained stable even under extreme conditions, such as a noise level of $\sigma = 1.0$ or the presence of 25 distractors.

Comparative error levels for the Input Map and Focus Map under varying numbers of distractors (left) and noise levels (right).

Key Insights

  • Spikes as Natural Filters: The inherent firing threshold of LIF neurons allows the network to ignore low-level luminance, acting as a built-in denoiser.
  • Synchrony and Continuity: The FM distinguishes moving objects from random noise by detecting synchronized activity and relying on the spatio-temporal continuity of real-world movement. Distractors that "pop" in and out fail to trigger a sustained response.
  • Efficiency: The event-based simulation only processes active neurons, allowing the system to process up to 100 images per second on standard 2008-era hardware.

Conclusion

The study successfully demonstrates that spiking neurons can implement robust attentional focusing. While the model showed slightly higher error than the Input Map at very low noise levels—likely due to discretization errors on the small 30x30 grid—its performance in high-noise environments is superior. Future work aims to integrate this mechanism into larger robotic frameworks for complex "covert attention" tasks.

Summary updated

BibTeX
@inproceedings{chevallier2008visual,
  author = {Sylvain Chevallier and Philippe Tarroux},
  title = {{Visual focus with spiking neurons}},
  booktitle = {ESANN},
  year = {2008},
  url = {https://openreview.net/forum?id=WweE-vjGi76},
}
Figure