Abstract
This paper explores modifications to a feedforward five-layer spiking convolutional network (SCN) of the ventral visual stream. The original model showed that a spike-timing-dependent plasticity (STDP) learning algorithm could perform unsupervised feature discovery. To study robustness, we replace non-leaky integrate-and-fire (IF) neurons with Izhikevich-like neurons and the original STDP rule with a novel probabilistic rule. The probabilistic STDP significantly improves performance for both neuron types. While Izhikevich-like neurons did not improve accuracy over IF neurons, performance remained comparable, demonstrating the model's robustness to more biologically realistic components.
Introduction: Bridging Biology and Bayes
In the quest to understand how the primate brain recognizes objects with such high speed and efficiency, researchers often utilize Spiking Neural Networks (SNNs). A landmark 2007 study by Masquelier and Thorpe demonstrated that a simple five-layer SNN using Spike-Timing-Dependent Plasticity (STDP) could learn to recognize visual features without explicit supervision. However, that original model relied on simplified "non-leaky integrate-and-fire" (IF) neurons and a heuristic STDP rule. This research addresses whether the model remains robust when using more complex, biologically realistic neurons and whether there is a rigorous mathematical foundation for STDP's effectiveness in feature discovery.
Model Architecture and Probabilistic Learning
The authors tested four distinct network architectures (SNN 1 through SNN 4) to isolate the effects of two major changes: the neuron model and the learning rule. The architecture follows a hierarchical structure typical of the ventral visual stream, moving from simple cells (S1) to complex cells (C2) before reaching a final classifier.
The primary innovation is the Probabilistic STDP rule. Unlike standard rules that utilize linear weight changes, this rule employs an exponential function of the current weight for Long-Term Potentiation (LTP). This allows synaptic weights to be interpreted as "log-odds" within a Bayesian framework. Essentially, the network performs a form of online Expectation Maximization (EM) to identify the most likely causes of visual input, bridging the gap between biological mechanisms and statistical learning.
Experimental Methodology and Datasets
The models were evaluated on two primary benchmarks: the Caltech dataset (focusing on faces and motorbikes) and the 3D-Object dataset (including categories like bicycles and monitors). To avoid "ceiling effects" where models perform too well to distinguish, the researchers introduced Gaussian noise to the test images to evaluate resilience.
Key Results and Insights
The experimental results yielded several critical insights:
- Superior Accuracy: SNN 2 (IF + Probabilistic STDP) and SNN 4 (Izhikevich + Probabilistic STDP) consistently outperformed their counterparts using the original learning rule. In face and motorbike recognition, the probabilistic rule showed significant improvements (p < 0.01).
- Biological Robustness: Switching to Izhikevich neurons—which possess more complex firing dynamics—did not degrade the model's performance. This suggests the high-level architecture of the ventral visual stream is robust enough to handle different underlying "hardware."
- Self-Limiting Weights: The probabilistic rule naturally converges, keeping weights within a specific range (0 to ~1.582) without requiring artificial "hard clipping."
- Early Learning Advantage: Izhikevich units tended to learn faster in the very early stages of training (200-300 iterations), though this advantage leveled off over time.
Conclusion and Future Work
The study demonstrates that the underlying reasons for stable performance in SNNs are preserved despite changes to explicit components. While Izhikevich neurons proved robustness, they did not significantly increase final accuracy compared to simpler IF neurons for basic classification. Future work could explore more complex temporal coding beyond the "first spike" to unlock higher performance in deeper spiking architectures.
Summary updated