Activation-wise Propagation: A One-Timestep Strategy for Spiking Neural Networks

Jian Song, Xiangfei Yang, Shangke Lyu, Donglin Wang

TL;DR

Researchers have developed Activation-wise Membrane Potential Propagation (AMP2), a novel strategy that allows Spiking Neural Networks (SNNs) to achieve high accuracy using only a single timestep. By mimicking the spatial diffusion of signals in biological brains, AMP2 eliminates the need for slow, iterative temporal updates, significantly reducing latency and memory consumption while setting new performance benchmarks for 3D point cloud and neuromorphic data.

What Problem Does This Paper Address?

Spiking Neural Networks (SNNs) are the "green" alternative to traditional Artificial Neural Networks (ANNs). They use discrete, sparse "spikes" to process information, which is incredibly energy-efficient on specialized neuromorphic hardware. However, SNNs have a major bottleneck: latency.

To reach high accuracy, standard SNNs typically rely on Timestep-Wise Updates (TWU). A neuron must iterate over multiple timesteps ($T$) to accumulate enough "membrane potential" (its internal hidden state) to fire a spike. This creates a painful trade-off: if you want high accuracy, you need a large $T$, which leads to high computational overhead, massive memory usage, and slow inference. The authors of this paper asked: Can we achieve high performance while eliminating these iterative updates entirely?

Methods

The authors propose AMP2, a unified hidden state update mechanism that shifts the burden of information accumulation from time to space. Instead of waiting for a single neuron to accumulate potential over several clock cycles, AMP2 allows information to flow through the network layers in a single pass.

The method consists of two core innovations:

  1. Activation/Neuron-Wise Propagation (AWP): Inspired by how molecules diffuse between neighboring neurons in the brain, AWP enables the direct propagation of membrane potentials between layers. The residual potential from one spiking layer is carried forward to initialize the next, facilitating rapid spatiotemporal integration.
  2. Residual Membrane Potential Connection (RMP): While standard residual connections (like those in ResNet) skip features or spikes, RMP introduces a dedicated shortcut for the membrane potentials themselves. This prevents "spike degradation" in deep networks, ensuring that the signal remains strong as it moves through the architecture.
Figure 1: Forward propagation of Spiking PointNet. Left: Timestep-Wise Update (TWU). Right: Activation-wise Membrane Potential Propagation (AMP2). AMP2 replaces timestep-wise iterative MP accumulation with neuron-wise accumulation, enabling an SNN that operates effectively at a single timestep.

Datasets

The researchers tested AMP2 across a wide variety of modalities to prove its general-purpose utility:

  • 3D Point Clouds: ModelNet40 (synthetic classification), ScanObjectNN (real-world classification), ShapeNetPart (part segmentation), and S3DIS (large-scale semantic segmentation).
  • Neuromorphic Data: DVS128Gesture, a benchmark for event-based gesture recognition captured by dynamic vision sensors.
  • Static Images: A 100-category subset of ImageNet (ImageNet100) to evaluate performance on traditional computer vision tasks.

Key Insights and Ideas

  • Trading Time for Space: The fundamental "aha" moment is that temporal accumulation isn't the only way to build up a signal. By propagating hidden states across layers (spatially), the network can "think" just as deeply in one timestep as a traditional SNN does in many.
  • Efficiency Gains are Massive: Moving from $T=4$ to $T=1$ isn't just a marginal improvement. The authors found that increasing timesteps from 1 to 4 results in a 2.5× increase in memory consumption and a 2.3× increase in training time. AMP2 recaptures this efficiency without sacrificing accuracy.
  • Universal Compatibility: Unlike many SNN optimizations that only work for specific architectures, AMP2 was successfully integrated into MLPs, CNNs, and even Spiking Transformers.
Figure 2: The workflow of Activation-wise Membrane Potential Propagation (AMP2) applied in Spiking PointNet.

Key Results

AMP2 delivers state-of-the-art (SOTA) results for point cloud SNNs, often outperforming models that use significantly more timesteps and parameters.

  • Point Cloud SOTA: On ModelNet40, an AMP2-based Spiking PointNet++ achieved 91.63% accuracy at $T=1$, the best result for point cloud SNNs to date.
  • Neuromorphic Leap: On the DVS128Gesture dataset, AMP2 boosted the accuracy of a single-timestep Spiking PointNet from a meager 31.22% to a robust 92.82%.
  • Hardware Speedup: When evaluated on actual neuromorphic hardware (Lynxi HP201), the inference time for a single batch dropped from ~2014ms (at $T=8$) to just 100.7ms with AMP2.
Figure 4: Impact of timesteps (1, 4, 8) on the inference time (seconds) of Spiking PointNet for a single batch, evaluated on the neuromorphic hardware Lynxi HP201.

Limitations and Future Work

While the results are impressive, the authors note a few areas for further exploration:

  • Scaling Up: The method needs to be evaluated on even larger-scale datasets like the full ImageNet-1k to see if the $T=1$ strategy holds up under extreme complexity.
  • Hardware Integration: Current neuromorphic hardware isn't yet fully optimized for these specific types of spatial propagation paths. The authors suggest that future collaboration with hardware designers could unlock even greater power savings.
  • Precise Power Measurement: The field still lacks standardized, precise power measurement tools for SNNs on diverse hardware, making it difficult to quantify the exact energy "win" in every scenario.

Summary updated

Abstract

Spiking neural networks (SNNs) have demonstrated significant potential in real-time multi-sensor perception tasks due to their event-driven and parameter-efficient characteristics. A key challenge is the timestep-wise iterative update of neuronal hidden states (membrane potentials), which complicates the trade-off between accuracy and latency. SNNs tend to achieve better performance with longer timesteps, inevitably resulting in higher computational overhead and latency compared to artificial neural networks (ANNs). Moreover, many recent advances in SNNs rely on architecture-specific optimizations, which, while effective with fewer timesteps, often limit generalizability and scalability across modalities and models. To address these limitations, we propose Activation-wise Membrane Potential Propagation (AMP2), a unified hidden state update mechanism for SNNs. Inspired by the spatial propagation of membrane potentials in biological neurons, AMP2 enables dynamic transmission of membrane potentials among spatially adjacent neurons, facilitating spatiotemporal integration and cooperative dynamics of hidden states, thereby improving efficiency and accuracy while reducing reliance on extended temporal updates. This simple yet effective strategy significantly enhances SNN performance across various architectures, including MLPs and CNNs for point cloud and event-based data. Furthermore, ablation studies integrating AMP2 into Transformer-based SNNs for classification tasks demonstrate its potential as a general-purpose and efficient solution for spiking neural networks.
BibTeX
@misc{song2025activation,
  author = {Jian Song and Xiangfei Yang and Shangke Lyu and Donglin Wang},
  title = {{Activation-wise Propagation: A One-Timestep Strategy for Spiking Neural Networks}},
  year = {2025},
  eprint = {2502.12791},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.12791},
}
Figure