Toward Robust Spiking Neural Network Against Adversarial Perturbation

Ling Liang, Kaidi Xu, Xing Hu, Lei Deng, Yuan Xie

Spiking Neural Networks (SNNs) are energy-efficient but highly vulnerable to adversarial attacks. This paper introduces S-IBP and S-CROWN, the first certified robust training framework specifically designed to handle the non-linear "firing" behavior and temporal dynamics of spiking neurons, significantly reducing attack vulnerability with minimal accuracy loss.

The Challenge of Securing SNNs

SNNs are often considered the "third generation" of neural networks, designed to mimic the brain's event-driven nature. They are incredibly efficient on neuromorphic hardware, but they share a fatal flaw with traditional Artificial Neural Networks (ANNs): adversarial vulnerability. A tiny, invisible change to the input—like flipping a few spikes or slightly altering pixel values—can cause the network to fail entirely.

Overview of SNN mechanics, showing spatial/temporal updates, membrane potential thresholds, and digital vs. spike inputs.

While researchers have developed "certified robustness" for ANNs—mathematical guarantees that a model will remain stable within a certain "noise" boundary—these methods do not translate directly to SNNs. SNNs use discrete, non-differentiable "firing" functions (Heaviside step functions) and maintain internal "memory" (membrane potential) that evolves over time. This paper bridges that gap, creating a way to calculate safety boundaries for models that operate in both space and time.

Methods: S-IBP and S-CROWN

The authors adapt the CROWN-IBP framework, a state-of-the-art robust training method, to accommodate the unique properties of SNNs:

  • S-IBP (Spiking Interval Bound Propagation): This technique tracks the "worst-case" range (interval) of neuron values as they propagate forward through the network layers.
  • S-CROWN: Because standard IBP bounds can be imprecise or "loose," S-CROWN employs linear relaxations to create tighter, more accurate upper and lower bounds for the non-linear firing function.
  • Temporal Bounding: Unlike standard networks, SNNs reset their state after firing. The authors formalized how to bound this "temporal update" across multiple time steps to prevent errors from accumulating.
Linear relaxations used in S-CROWN to bound the non-differentiable step function and temporal resets.

Dual Input Formalization

SNNs can process two distinct types of data, and the authors formalized boundaries for both:

  1. Digital Inputs: Handled via $\ell_\infty$-norm, which accounts for small, continuous changes to pixel intensity.
  2. Spike Inputs: Handled via $\ell_0$-norm, which limits the discrete number of individual spikes that an attacker can flip.

Key Insights and Performance

The researchers evaluated their methods on three standard benchmarks: MNIST (Digital), FMNIST (Digital), and NMNIST (Neuromorphic MNIST, representing true spike-based temporal data).

The "Unstable" Neuron Challenge

The most difficult part of certifying an SNN occurs when a neuron's potential is hovering right at the firing threshold. The authors' linear relaxation strategy specifically targets these "unstable" cases to prevent the mathematical bounds from "exploding" and rendering the certification useless. Furthermore, because errors in SNNs accumulate over time, bounding the temporal reset mechanism is critical to ensure the model doesn't drift into an incorrect state over several milliseconds of processing.

Results

The proposed robust training significantly improved model security across all datasets. On the MNIST dataset, the framework achieved a 37.7% reduction in attack error. Crucially, this security boost came at a cost of only a 3.7% loss in original (clean) accuracy. The study also found that Convolutional (CONV) SNNs generally achieve better robustness than Fully Connected (FC) ones, suggesting that the spatial hierarchy of CONV layers helps "absorb" the noise introduced by the bounding process. Finally, they discovered that using 3 "robust training time steps" ($T'$) provided the optimal balance between tight bounds and computational efficiency.

Limitations and Future Work

While groundbreaking, the approach has limitations. Robust training is computationally expensive; depending on the settings, it can be 2.6x to 12.9x slower than standard training per epoch. Additionally, the framework is highly sensitive to the hyperparameter $\epsilon$ (the size of the allowed perturbation). If $\epsilon$ is too large, the training may diverge; if too small, the robustness gains are negligible. Scaling these certified bounds to larger, real-world neuromorphic datasets—such as those used in autonomous driving—remains a significant challenge for the field.

Summary updated

Abstract

As spiking neural networks (SNNs) are deployed increasingly in real-world efficiency critical applications, the security concerns in SNNs attract more attention. Currently, researchers have already demonstrated an SNN can be attacked with adversarial examples. How to build a robust SNN becomes an urgent issue. Recently, many studies apply certified training in artificial neural networks (ANNs), which can improve the robustness of an NN model promisely. However, existing certifications cannot transfer to SNNs directly because of the distinct neuron behavior and input formats for SNNs. In this work, we first design S-IBP and S-CROWN that tackle the non-linear functions in SNNs' neuron modeling. Then, we formalize the boundaries for both digital and spike inputs. Finally, we demonstrate the efficiency of our proposed robust training method in different datasets and model architectures. Based on our experiment, we can achieve a maximum $37.7\%$ attack error reduction with $3.7\%$ original accuracy loss. To the best of our knowledge, this is the first analysis on robust training of SNNs.
BibTeX
@inproceedings{liang2022toward,
  author = {Ling Liang and Kaidi Xu and Xing Hu and Lei Deng and Yuan Xie},
  title = {{Toward Robust Spiking Neural Network Against Adversarial Perturbation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=Ncyc0JS7Q16},
}
Figure