Mapping Continuous Neural Networks to Discrete Spiking Architectures

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2132 papers159 conceptsfrozen 2026-10-04

Spiking Neural Networks (SNNs) are often called the "third generation" of neural networks because they mimic the brain's event-driven communication. Unlike standard Artificial Neural Networks (ANNs) that pass continuous values, SNNs communicate via discrete, binary pulses (spikes) over time. This sparsity allows neuromorphic hardware to achieve orders of magnitude better energy efficiency than GPUs.

However, training SNNs directly is notoriously difficult because the spike function is non-differentiable, breaking the standard backpropagation algorithm. ANN-to-SNN conversion bypasses this "dead neuron" problem by training a high-performance model in the continuous domain and then mapping its parameters to a spiking architecture.

The Core Intuition: Rate Coding and Thresholds

The fundamental idea behind conversion is that the continuous activation of a ReLU neuron in an ANN can be represented as the firing rate of a spiking neuron. If an ANN neuron outputs a value of 0.7, we want the corresponding spiking neuron to fire in 70% of its available time steps.

To achieve this, researchers like Peter O’Connor in Deep Spiking Networks demonstrated that Integrate-and-Fire (IF) neurons can be made functionally equivalent to ReLU units. The weights from the ANN are transferred directly to the SNN, but a critical challenge arises: normalization. If the weights are too large, the spiking neurons saturate (fire every step); if they are too small, they never fire.

Yangfan Hu and Huajin Tang addressed this in Spiking Deep Residual Network by scaling weights and thresholds layer-by-layer to ensure the maximum activation of the ANN corresponds to the maximum firing rate of the SNN. This allowed the conversion of deep architectures like ResNet-110 with minimal accuracy loss.

Beyond ReLU: Converting Transformers and LLMs

While early conversion focused on simple CNNs, modern research has expanded to complex architectures like Vision Transformers (ViTs) and Large Language Models (LLMs). These models contain non-linearities like Softmax, LayerNorm, and GeLU that do not map naturally to simple firing rates.

To solve this, Sangwoo Hwang introduced SpikedAttention in SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion. Instead of approximating the Softmax value directly, they use "Winner-Oriented Spike Shift" to prioritize the most important attention scores in the temporal domain. Similarly, Xinzhe Yuan developed Plug-and-Play Spiking Operators, which decompose complex functions into basic primitives like division and exponentiation that can be executed by populations of spiking neurons.

Reducing Latency: From Rates to Timing

The biggest drawback of traditional rate-based conversion is latency. To accurately represent a continuous value like 0.714, a neuron might need 1,000 time steps, which negates the speed benefits of SNNs.

Recent work focuses on "temporal coding" to achieve near-lossless conversion in fewer than 10 steps:

  1. Time-to-First-Spike (TTFS): Instead of counting spikes, the information is encoded in when the first spike occurs. Lusen Zhao and Zhaofei Yu used this in TTFSFormer to convert Transformers with almost zero accuracy drop.
  2. Differential Coding: Zihan Huang and Wei Fang proposed Differential Coding for Training-Free ANN-to-SNN Conversion, where neurons transmit changes in information. This allows the network to react instantly to new inputs, slashing the number of required time steps.
  3. Burst Spiking: Yang Li introduced Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes, allowing neurons to fire multiple spikes in a single time step to convey more information quickly.

Why Conversion Works: Neurons as Optimizers

A theoretical breakthrough by Hyunseok Oh in Sign Gradient Descent-based Neuronal Dynamics suggests that spiking neurons are not just passive filters; they are essentially performing subgradient descent to minimize the approximation error between the SNN's output and the ANN's target activation.

This perspective explains why conversion is so robust: the membrane potential of a spiking neuron acts as an error accumulator. If the neuron hasn't fired enough to match the ANN's value, the potential builds up until a spike is triggered, naturally correcting the discretization error over time.

Real-World Deployment

The ultimate goal of this research is to move AI from power-hungry GPUs to the edge. By converting pre-trained models, researchers have successfully deployed:

  • Object Tracking: Caihong Yuan developed SiamSNN for real-time tracking on IBM's TrueNorth chip.
  • Robotics: Kenneth Stewart demonstrated 20x better energy efficiency for robot control on Intel's Loihi 2 hardware.
  • Scientific Discovery: Nicholas Pritchard applied conversion to RFI detection in radio astronomy, eliminating expensive memory searches.

By treating SNNs as a temporal discretization of well-understood ANNs, the field has turned the "black box" of biological signaling into a precise engineering tool for green AI.

Papers in this version

Showing 60 of 132 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 107 papers
  2. Version 1 · 2026-10-04 · 132 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_ann_to_snn_conversion_116bea56,
  title = {ANN-to-SNN conversion},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/ann-to-snn-conversion/rdr_116bea56575e4bcca4f1fc582cba0218},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_mapping_continuous_neural_networks_to_di_116bea56_v1,
  title = {Mapping Continuous Neural Networks to Discrete Spiking Architectures},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mapping-continuous-neural-networks-to-discrete-spiking-architectures/rdr_116bea56575e4bcca4f1fc582cba0218/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}