First-spike coding promotes accurate and efficient spiking neural networks for discrete events with rich temporal structures

Siying Liu, Vincent C. H. Leung, Pier Luigi DragottiFrontiers in Neuroscience 2023 · DOI

TL;DR

Researchers have developed a novel "First-Spike" (FS) coding strategy for Spiking Neural Networks (SNNs) that prioritizes the timing of the initial spike for decision-making rather than just counting total spikes. This approach achieves classification accuracy comparable to traditional rate-coding while significantly improving energy efficiency and better capturing the complex temporal patterns found in real-world event-based data.

What Problem Does This Paper Address?

Most current Spiking Neural Networks rely on Rate Coding (FR), where the network makes decisions based on how many times a neuron fires within a window. While effective, this ignores the precise timing of spikes—the very thing that makes biological brains so efficient.

An alternative is Time-to-First-Spike (TTFS) coding, which is theoretically faster and more efficient. However, traditional TTFS methods are notoriously difficult to train and often impose a "one-spike constraint," meaning each neuron can only fire once. This is unrealistic for real-world event-based data (like streams from a neuromorphic camera), where a single feature might trigger multiple spikes over time.

The authors address this gap by proposing First-Spike (FS) coding. Unlike TTFS, it doesn't restrict neurons to a single spike, but it focuses the learning process on the timing of the very first spike in the output layer to make a classification decision.

Methods

The core of the paper is a new supervised learning framework designed to handle discrete spike trains:

  • Decision Strategy: In the output layer, the neuron that fires first determines the predicted class. If no neurons fire, the one with the highest membrane potential is chosen.
  • Surrogate Gradient Learning: Since the "step function" of a spike is non-differentiable (making standard backpropagation impossible), the authors use a surrogate gradient.
  • Gaussian Error Assignment: This is the "secret sauce." To train the network on spike timing, they propagate the error from the first-spike time back to multiple spikes through a Gaussian window. This allows the network to shift spike timings gradually during training rather than just toggling them on or off.
  • Training Strategies:
    • Empty Sequences: They append "silence" to the end of training data. This gives "lazy" neurons more time to fire during the learning phase, preventing them from becoming permanently inactive.
  • Layer-Specific Hyperparameters: They found that using small time constants ($\tau$) and thresholds ($\theta$) in early layers (to catch quick features) and larger ones in the final layer (to accumulate evidence before deciding) yielded the best results.

Datasets

The authors specifically avoided "static" datasets like N-MNIST, arguing they don't contain enough temporal information. Instead, they used:

  • DVSGesture: Hand gestures recorded with a DVS camera (repetitive temporal structure).
  • DVSPlane: Recordings of toy airplanes (complex, non-repetitive temporal structure).
  • SHD (Spiking Heidelberg Dataset): Spoken digits converted to spikes (rich audio temporal structure).
  • N-TIDIGITS: Another spiking audio dataset of spoken numbers.

Key Insights and Ideas

  • The "Evidence Accumulation" Trade-off: There is a clear trade-off between speed and accuracy. Interestingly, a longer delay in the first spike often led to higher accuracy. This suggests the network is "waiting" to accumulate enough temporal evidence to make a more reliable call.
  • Sparsity as a Feature: FS coding naturally encourages sparsity. In an FR (rate-coded) network, non-target neurons might keep firing uselessly. In an FS network, once the first spike is optimized, the non-target neurons are actively encouraged not to fire early, leading to much lower overall spike counts.
  • Temporal vs. Spatial Strengths: FS coding is particularly superior for audio data (SHD/N-TIDIGITS) where the information is purely temporal. For visual data with repetitive patterns (DVSGesture), the benefits over rate-coding are less pronounced because the spatial information dominates.

Key Results

  • Accuracy: FS coding achieved comparable accuracy to FR coding across all datasets. For example, on the SHD dataset, FS coding reached 87.6% accuracy compared to 77.8% for FR coding under similar conditions.
  • Efficiency: FS models were significantly more energy-efficient. On audio tasks, the FS models produced roughly 3x to 4x fewer spikes than their rate-coded counterparts.
  • Robustness: When using Adaptive Leaky Integrate-and-Fire (AdLIF) neurons, the FS model achieved state-of-the-art results on the SHD dataset (94.08%), outperforming many complex recurrent architectures.

Limitations and Future Work

  • Inactive Neurons: Training FS networks is still more sensitive than rate-coded ones. If a neuron never fires, it receives no gradient, making "dead neurons" a persistent challenge that requires careful initialization and the "empty sequence" trick.
  • Latency: While FS coding is efficient, the "evidence accumulation" insight means these networks aren't always "faster" in terms of wall-clock time if they need to wait for a late first spike to be accurate.
  • Future Directions: The authors suggest exploring ways to reduce this temporal delay without sacrificing accuracy and applying these FS strategies to even deeper, more complex architectures like Spiking Transformers.

Summary updated

Abstract

Spiking neural networks (SNNs) are well-suited to process asynchronous event-based data. Most of the existing SNNs use rate-coding schemes that focus on firing rate (FR), and so they generally ignore the spike timing in events. On the contrary, methods based on temporal coding, particularly time-to-first-spike (TTFS) coding, can be accurate and efficient but they are difficult to train. Currently, there is limited research on applying TTFS coding to real events, since traditional TTFS-based methods impose one-spike constraint, which is not realistic for event-based data. In this study, we present a novel decision-making strategy based on first-spike (FS) coding that encodes FS timings of the output neurons to investigate the role of the first-spike timing in classifying real-world event sequences with complex temporal structures. To achieve FS coding, we propose a novel surrogate gradient learning method for discrete spike trains. In the forward pass, output spikes are encoded into discrete times to generate FS times. In the backpropagation, we develop an error assignment method that propagates error from FS times to spikes through a Gaussian window, and then supervised learning for spikes is implemented through a surrogate gradient approach. Additional strategies are introduced to facilitate the training of FS timings, such as adding empty sequences and employing different parameters for different layers. We make a comprehensive comparison between FS and FR coding in the experiments. Our results show that FS coding achieves comparable accuracy to FR coding while leading to superior energy efficiency and distinct neuronal dynamics on data sequences with very rich temporal structures. Additionally, a longer time delay in the first spike leads to higher accuracy, indicating important information is encoded in the timing of the first spike.
BibTeX
@inproceedings{liu2023first,
  author = {Siying Liu and Vincent C. H. Leung and Pier Luigi Dragotti},
  title = {{First-spike coding promotes accurate and efficient spiking neural networks for discrete events with rich temporal structures}},
  booktitle = {Frontiers in Neuroscience},
  year = {2023},
  doi = {10.3389/fnins.2023.1266003},
}
Figure