Efficient single input-output layer spiking neural classifier with time-varying weight model

Abeegithan Jeyasothy, Savitha Ramasamy, Suresh Sundaram

Introduction

Spiking Neural Networks (SNNs) are biologically inspired models that process information through the precise timing of pulses, or spikes. While theoretically more powerful than traditional artificial neural networks, SNNs are notoriously difficult to train. Standard backpropagation often fails due to the non-differentiable nature of spikes, leading to "silent neurons" that block gradient flow. Furthermore, most SNNs rely on static synaptic weights, requiring deep, computationally expensive architectures to solve complex non-linear problems.

To address these challenges, researchers have developed SEF-M (Synaptic Efficacy Function with Meta-neuron based learning), a supervised learning algorithm that shifts complexity from network depth to the temporal dynamics of individual synapses.

Methodology: The SEF-M Architecture

The core innovation of SEF-M is the use of Time-Varying Weights (SEF). Instead of a single scalar value, the connection between neurons is modeled as a function $w(t)$, constructed by summing multiple Gaussian functions centered at the times of presynaptic spikes. This allows the synaptic strength to fluctuate precisely in response to temporal input patterns.

The SEF-M architecture featuring time-varying weights in a single-layer network.

Unlike traditional deep SNNs, SEF-M utilizes a compact single input-output layer architecture. By making synapses "smarter" through time-dependency, the network can solve non-linear classification tasks without hidden layers. The learning process is guided by a Meta-neuron Learning Rule, which adjusts the amplitude of Gaussian components based on the error between the actual postsynaptic potential and the desired firing threshold. Additionally, a "skip sample" strategy is employed to update weights only when necessary, significantly enhancing computational efficiency.

Experimental Results and Discussion

The SEF-M model was rigorously evaluated against state-of-the-art SNN algorithms, including SpikeProp, SWAT, and TMM-SNN, across ten benchmark datasets from the UCI Machine Learning Repository.

Comparative performance of SEF-M against other SNN algorithms on UCI benchmark datasets.

As shown in the results, SEF-M consistently outperformed or matched existing methods. Notably, on the Mammogram dataset, it achieved a testing accuracy of 93.6%, surpassing the next best algorithm by approximately 8%. Statistical analyses, including Friedman tests and Fisher’s LSD post-hoc analysis, confirmed these improvements are significant within a 95%–99% confidence interval.

An ablation study on the JAFFE facial expression dataset further highlighted the importance of the time-varying weight model, which provided a 14% accuracy boost over a static weight baseline (99.93% training and 89.14% testing accuracy for the time-varying model).

Conclusion and Future Directions

SEF-M demonstrates that temporal richness in synapses can compensate for a lack of architectural depth, leading to efficient and accurate SNNs. However, the model's performance is sensitive to hyperparameters like the efficacy update range ($\sigma$) and the chosen input encoding scheme. Future work will focus on automating parameter tuning and testing the scalability of this single-layer approach on larger, high-dimensional datasets such as ImageNet.

Summary updated

Abstract

This paper presents a supervised learning algorithm, namely, the Synaptic Efficacy Function with Meta-neuron based learning algorithm (SEF-M) for a spiking neural network with a time-varying weight model. For a given pattern, SEF-M uses the learning algorithm derived from meta-neuron based learning algorithm to determine the change in weights corresponding to each presynaptic spike times. The changes in weights modulate the amplitude of a Gaussian function centred at the same presynaptic spike times. The sum of amplitude modulated Gaussian functions represents the synaptic efficacy functions (or time-varying weight models). The performance of SEF-M is evaluated against state-of-the-art spiking neural network learning algorithms on 10 benchmark datasets from UCI machine learning repository. Performance studies show superior generalization ability of SEF-M. An ablation study on time-varying weight model is conducted using JAFFE dataset. The results of the ablation study indicate that using a time-varying weight model instead of single weight model improves the classification accuracy by 14%. Thus, it can be inferred that a single input-output layer spiking neural network with time-varying weight model is computationally more efficient than a multi-layer spiking neural network with long-term or short-term weight model.
BibTeX
@misc{jeyasothy2019efficient,
  author = {Abeegithan Jeyasothy and Savitha Ramasamy and Suresh Sundaram},
  title = {{Efficient single input-output layer spiking neural classifier with time-varying weight model}},
  year = {2019},
  eprint = {1904.10400},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1904.10400},
}
Figure