Abstract
There is an increasing demand to process streams of temporal data in energy-limited scenarios such as embedded devices, driven by the advancement and expansion of Internet of Things (IoT) and Cyber-Physical Systems (CPS). Spiking neural network has drawn attention as it enables low power consumption by encoding and processing information as sparse spike events, which can be exploited for event-driven computation. Recent works also show SNNs' capability to process spatial temporal information. Such advantages can be exploited by power-limited devices to process real-time sensor data. However, most existing SNN training algorithms focus on vision tasks and temporal credit assignment is not addressed. Furthermore, widely adopted rate encoding ignores temporal information, hence it's not suitable for representing time series. In this work, we present an encoding scheme to convert time series into sparse spatial temporal spike patterns. A training algorithm to classify spatial temporal patterns is also proposed. Proposed approach is evaluated on multiple time series datasets in the UCR repository and achieved performance comparable to deep neural networks.
Introduction
As the Internet of Things (IoT) and Cyber-Physical Systems (CPS) continue to expand, there is a massive demand for processing temporal sensor data on energy-constrained embedded devices. Spiking Neural Networks (SNNs) have emerged as a promising solution because they mimic the brain's energy efficiency, processing information via sparse "spikes" rather than continuous values. This event-driven nature allows for significantly lower power consumption compared to traditional artificial neural networks.
However, SNNs face three major hurdles in time series tasks. First, most SNNs use "rate coding," which represents values by the number of spikes in a window, often ignoring precise timing and failing to handle the varying scales of multivariate sensor data. Second, the output of a spiking neuron is a non-differentiable step function, making standard backpropagation impossible to apply directly. Finally, existing models often struggle with temporal credit assignment, making it difficult to learn long-term dependencies across time steps efficiently.
Methodology
This research proposes a framework to overcome these challenges through three primary innovations: a novel encoding scheme, an iterative model formulation, and surrogate gradient training.
Spatial-Temporal Population Encoding
Instead of simple rate coding, the framework utilizes a population of "Current-based Integrate and Fire" (CUBA) neurons to encode each signal. Each neuron in the population is configured with different parameters, such as gain and time constants. This diversity allows the group to capture complex temporal patterns and handle negative values or varying signal ranges with very few spikes, ensuring high information density.
Iterative SNN Model and Training
The authors derive a discrete-time, iterative version of the Spike Response Model (SRM). This formulation allows the SNN to be "unrolled" in time, similar to a Recurrent Neural Network (RNN), making it compatible with Backpropagation Through Time (BPTT). To solve the non-differentiability problem, a smooth sigmoid function is used as a "surrogate" for the spike function during the backward pass of training. This allows gradients to flow through the network while keeping the forward pass purely spike-based and event-driven.
Experimental Setup
The approach was evaluated using multiple multivariate time series (MTS) datasets from the UCR/UEA repository. These include diverse tasks such as Articulary Word Recognition, FaceDetection, BasicMotions, Heartbeat, Spoken Arabic Digits, JapaneseVowels, and RacketSports. Efficiency testing was specifically conducted on the Atrial Fibrillation dataset to measure the impact of the proposed encoding on computational load.
Results and Discussion
The proposed SNN framework achieved classification accuracy comparable to state-of-the-art deep learning models while significantly reducing computational overhead and model size.
- Comparable Accuracy: On the Articulary Word Recognition task, the SNN achieved an accuracy of 98.27%, nearly identical to a much larger LSTM (98.31%) and RNN (98.20%).
- Massive Parameter Reduction: The SNN model used only 125,880 parameters, which is roughly 11% of the parameters required by a comparable LSTM (1.1 million parameters).
- Efficiency through Sparsity: The spatial-temporal encoding is incredibly sparse. For Articulary Word Recognition, the method reduced the input from 1,296 spike trains (required by rate coding) to just 45 without losing accuracy. On the Atrial Fibrillation dataset, the spike count dropped from over 23,000 in rate coding to just 164.5, representing a massive reduction in inference operations.
Conclusion
This work demonstrates that by focusing on the structure and timing of spikes, SNNs can represent multivariate time series data more densely and accurately than traditional methods. While the current implementation uses surrogate gradients on standard GPUs, the framework is designed to exploit the event-driven nature of specialized neuromorphic hardware. Future work will explore biologically plausible local learning rules and scaling the framework to even larger multivariate datasets.
Summary updated