TL;DR
The TS-LIF (Temporal Segment Leaky Integrate-and-Fire) model introduces a biologically inspired, dual-compartment spiking neuron designed to master multi-scale time series forecasting. By separating input signals into low- and high-frequency components across dendritic and somatic branches, it achieves superior accuracy and remarkable robustness to missing data compared to traditional spiking and artificial neural networks.
What Problem Does This Paper Address?
Time series forecasting is everywhere—from predicting stock market swings to managing power grids. While deep learning models like Transformers and Temporal Convolutional Networks (TCNs) are highly effective, they are computationally expensive. Spiking Neural Networks (SNNs) offer a more energy-efficient, "event-driven" alternative inspired by the human brain.
However, the standard workhorse of SNNs, the Leaky Integrate-and-Fire (LIF) neuron, has a major flaw: it "forgets" too quickly. Its rapid membrane potential decay makes it struggle with long-term dependencies and complex signals that contain both slow-moving trends (low frequency) and sudden fluctuations (high frequency). Existing multi-compartment models have tried to fix this but often lack a clear functional distinction between parts of the neuron, leading to information loss during processing.
Methods
The authors propose the TS-LIF neuron, which mimics the structural complexity of biological neurons more closely than a simple LIF model.
- Dual-Compartment Architecture: The neuron is split into a dendritic compartment and a somatic compartment.
- The dendrite acts as a low-pass filter, focusing on long-term trends.
- The soma acts as a high-pass filter, capturing rapid, short-term changes.
- Direct Somatic Injection: Unlike previous models where all input had to pass through the dendrite first (risking information loss), TS-LIF allows current to be injected directly into the soma.
- Multi-Scale Spike Generation: Both compartments can generate spikes, which are then combined into a weighted "mixed output." This allows the network to preserve features from different timescales simultaneously.
The model also includes learnable parameters that allow the network to automatically tune its frequency response and the balance between its compartments during training.
Datasets
The model was evaluated on four widely-used multivariate time series benchmarks:
- Metr-la & Pems-bay: Traffic speed data from Los Angeles and the Bay Area.
- Solar: Solar power production records.
- Electricity: Hourly electricity consumption data.
- Delayed Spiking XOR: A synthetic benchmark specifically designed to test a model's ability to retain information over long delays.
Key Insights and Ideas
- Functional Heterogeneity is Key: The core breakthrough is that by forcing different parts of the neuron to specialize in different frequencies, the network becomes much better at decomposing complex signals.
- Frequency Separation in Action: Temporal analysis reveals that while older multi-compartment models produce messy, mixed-frequency responses, TS-LIF clearly separates slow "waves" into the dendrite and fast "jitters" into the soma.
- Biological Realism Pays Off: Features like axosomatic synapses (direct connections to the soma) and dendritic spikes aren't just biological trivia; they directly mitigate the "vanishing gradient" problem and improve information extraction across scales.
Key Results
TS-LIF was tested across various backbones (TCN, GRU, and Transformer) and consistently outperformed both vanilla SNNs and traditional Artificial Neural Networks (ANNs).
- Superior Accuracy: In the Electricity dataset, the TS-former achieved a significant reduction in Root Relative Squared Error (RSE) compared to the state-of-the-art iSpikformer.
- Robustness to Missing Data: Even when 80% of the input data was missing, TS-LIF models maintained high accuracy. For instance, on the Electricity dataset with 80% missing values, TS-GRU improved the $R^2$ score by roughly 20% over the baseline Spike-GRU.
- Efficiency: The model maintains the energy-efficient "spike-based" nature of SNNs while achieving performance competitive with heavy-duty Transformer models.
Limitations and Future Work
While TS-LIF excels at temporal modeling (the "time" part of time series), the authors acknowledge it currently lacks explicit mechanisms to handle cross-variable correlations (how one variable affects another in multivariate data).
Future research directions include:
- Developing SNN mechanisms to capture these inter-variable relationships more effectively.
- Expanding the TS-LIF framework to other complex tasks like anomaly detection, classification, and generative modeling.
Summary updated