Abstract
This paper introduces a distributed Spiking Neural Network (SNN) designed to extract visual saliency from robot camera images. By leveraging the temporal dynamics of spikes and a novel "Gaussian difference" model for post-synaptic potentials, the system achieves biologically inspired saliency detection where the order of firing naturally ranks the importance of visual features.
Introduction
In computer vision and robotics, "saliency" refers to identifying the most conspicuous parts of an image—the areas that should grab an agent's attention first. While classical models like the Itti model are effective, they often rely on complex "winner-takes-all" algorithms and inhibition-of-return mechanisms to rank salient locations. Biological systems, however, utilize temporal advantages that these classical methods miss.
Spiking Neural Networks (SNNs), which mimic discrete neuronal pulses, offer a biologically plausible alternative but are often computationally expensive. This research addresses two primary challenges: building an SNN architecture that extracts saliency effectively and ensuring the system is computationally efficient for potential real-time robotic applications.
Methods
The proposed distributed architecture is composed of 2D neural maps. The system utilizes a Two-Pathway Architecture inspired by the parvocellular and magnocellular pathways in primates, splitting processing into high-frequency (fine detail) and low-frequency (rough signal) channels. Saliency is detected only when a neuron receives simultaneous spikes from both channels, acting as a coincidence detector.
To improve efficiency, the authors implemented a Gaussian Difference PSP Model. Unlike standard alpha-functions, this model allows the contribution of multiple spikes to be calculated in a single step. Furthermore, the network is distributed across multiple computing nodes using an event-driven approach, processing only active neurons. To save bandwidth, nodes use Connection Masks, where a single spike message triggers a locally generated spatial pattern of impact.
Experimental Setup
The system was tested using real-world images acquired from a robot in a laboratory environment. Original 320x240 pixel images were downsampled to 76x56 pixels for processing. This study focused exclusively on luminance (brightness) information.
Results and Key Insights
The SNN produced saliency maps that were qualitatively identical to classical benchmarks using Gabor and Difference of Gaussians filters. A significant advantage of this approach is that temporal ranking is inherent: the most salient points fire first. This "rank order coding" provides an automatic classification of importance based on time, eliminating the need for additional ranking algorithms.
By requiring coincidence between high-frequency and low-frequency pathways, the model filters noise and focuses on structural details in high-contrast regions. Processing pixels from brightest to darkest further enhances efficiency by triggering early spikes for important features and potentially ignoring dark, non-salient areas.
Discussion and Future Work
While the distributed system successfully managed over 100,000 neurons, synchronization overhead between nodes remains a hurdle for real-time performance compared to classical methods. Future work aims to incorporate color and orientation features to match the robustness of state-of-the-art models. Additionally, transitioning from software simulation to dedicated neuromorphic hardware could provide the speed necessary for real-time robotics.
Summary updated