Leveraging spiking deep neural networks to understand neural mechanisms underlying selective attention

Lynn K. A. Sörensen, Davide Zambrano, Heleen A. Slagter, Sander M. Bohté, H. Steven ScholtebioRxiv 2020 · DOI

TL;DR

This paper uses spiking deep neural networks (sDCNNs) to investigate how different computational mechanisms of attention (gain modulation and precision) affect performance on a visual search task. They found that connection gain, which modulates the output of neurons, was the most effective mechanism for improving performance and replicating key findings from neuroscience.

What Problem Does This Paper Address?

Spatial attention is crucial for our ability to focus on relevant information in a cluttered environment. While we know it improves performance, the precise neural mechanisms by which it achieves this are still debated. Theories suggest attention might work by amplifying signals (gain modulation) or reducing internal noise (precision modulation). Existing computational models have often simplified these mechanisms or used artificial neuron models, making it difficult to directly compare these theories and link them to biological findings. This paper aims to bridge this gap by using more biologically realistic spiking neural networks to test these different attention mechanisms on a complex visual task.

Methods

The researchers employed spiking deep convolutional neural networks (sDCNNs), which are designed to mimic biological neurons more closely than standard deep learning models. They implemented three distinct attention mechanisms:

  • Input Gain: Modulating the incoming signal to a neuron.
  • Connection Gain: Modulating the outgoing signal (synaptic weight) from a neuron.
  • Precision: Modulating the internal noise of a neuron, effectively changing the reliability of its output.

These mechanisms were integrated into a ResNet-18 architecture. The sDCNNs were trained on a large image dataset and then fine-tuned for a visual search task. The researchers then analyzed the network's performance (target discrimination, detection times) and internal activity (evoked potentials, firing rates, latencies) under different attention conditions. They also used representational similarity analysis (RSA) to understand how attention altered the network's internal representations.

Datasets

  • ImageNet: Used for pre-training the ResNet-18 architecture.
  • Custom Visual Search Dataset: Curated from the COCO dataset, consisting of street scenes and food scenes with specific target categories. This dataset was designed to be challenging and naturalistic, featuring complex contexts and multiple potential target objects.

Key Insights and Ideas

  • Connection Gain is King: Modulating the output connections of neurons (connection gain) proved to be the most effective mechanism for improving performance on the visual search task, mirroring human behavior in similar tasks.
  • Asymmetry Matters: The sigmoidal activation function of biological neurons, which is asymmetric, plays a crucial role in the effectiveness of gain mechanisms. Connection gain, which leverages this asymmetry, was more effective than input gain.
  • Gain Enhances, Precision Suppresses: Gain-based mechanisms (input and connection gain) were found to add task-relevant information to the network's representations, effectively enhancing the signal. Precision, on the other hand, primarily worked by suppressing internal noise.
  • Spiking Networks Bridge the Gap: Using sDCNNs allowed the researchers to not only observe performance changes but also to link them to biologically plausible neural changes (evoked potentials, firing rates, latencies), providing a more comprehensive understanding of attention.

Key Results

  • Performance: Connection gain led to the largest improvements in target discrimination and the fastest detection times, significantly outperforming precision and input gain.
  • Neural Activity: All attention mechanisms modulated evoked potentials, firing rates, and latencies in ways that qualitatively mirrored findings from electrophysiological studies in animals. Connection gain showed the most pronounced effects.
  • Representational Similarity: Gain mechanisms enhanced categorical information in the network's representations, while precision primarily reduced noise.

Limitations and Future Work

  • No Synaptic Noise: The sDCNN models used did not incorporate synaptic noise, which is a known source of inter-trial variability in biological neurons. Future work could include synaptic noise to better replicate certain aspects of neural processing.
  • Architecture Specificity: The findings are based on a ResNet-18 architecture. Future studies could explore how these results generalize to other network architectures.
  • Dynamic Modulation: The study focused on static attentional modulation. Future research could investigate how the strength and timing of attention dynamically change throughout the visual hierarchy.
  • Weight Adaptation: The researchers speculate that precision might be more effective in models that allow for weight adaptation after attentional selection. This is a promising avenue for future investigation.

Summary updated

Abstract

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. The specific mechanisms linking neural changes to changes in performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effects of noise suppression (precision) and two different gain modulation mechanisms on performance on a visual search task with complex real-world images. Unlike standard artificial neurons, biological neurons have saturating activation functions, permitting implementation of attentional gain as gain on a neuron's input or on its outgoing connection.
BibTeX
@misc{sorensen2020leveraging,
  author = {Lynn K. A. Sörensen and Davide Zambrano and Heleen A. Slagter and Sander M. Bohté and H. Steven Scholte},
  title = {{Leveraging spiking deep neural networks to understand neural mechanisms underlying selective attention}},
  year = {2020},
  doi = {10.17605/osf.io/6tpz8},
  howpublished = {bioRxiv},
}
Figure