Spiking neural networks are neural networks whose neurons communicate with binary pulses ("spikes") rather than real numbers. They can run on neuromorphic chips at very low power, because a spike triggers an addition instead of a multiplication. The most practical way to get an accurate spiking network is ANN-to-SNN conversion: train an ordinary network, then replace each activation with a neuron that fires a number of spikes proportional to that activation. A known result makes this exact. A network whose activations are quantized to $b$ bits behaves identically to a spiking network that runs for $T = 2^b - 1$ time steps (Fast-SNN). Every conversion method in our file, however, gives all layers the same number of time steps. Under that equivalence, a single global time budget amounts to uniform quantization. The quantization literature has shown for years that uniform quantization wastes resources, because some layers are badly damaged by coarse rounding and others barely notice (HAWQ).
We propose to give each layer its own spike budget, the maximum number of spikes a neuron in that layer may emit. Budgets would be chosen by how sensitive the loss is to rounding that layer's activations, and the cost of each budget would be measured in spiking-hardware terms rather than bits. The central observation is that cost behaves differently in a spiking network. Adding one bit of activation precision to an ordinary accelerator costs memory that grows with $\log_2$ of the number of levels. Adding one quantization level to a spiking layer costs an extra time step and a share of extra spikes, each fanned out to every downstream synapse. The cost therefore grows roughly linearly in the number of levels and is weighted by connectivity. Methods that allocate bits for ordinary networks optimize the wrong price. We also propose an execution order that makes heterogeneous budgets exact while using only ordinary on/off spikes. This removes the negative spikes that recent low-latency conversion methods rely on and that standard neuromorphic chips cannot emit.
Core Claim
At a matched hardware cost, a converted spiking network whose per-layer spike budgets are allocated by activation sensitivity, with a spike-count cost model, will be more accurate than (a) a uniform-budget conversion and (b) a conversion whose budgets come from a conventional bit-cost mixed-precision allocator. Two cost measures are used for matching: synaptic operations and membrane-state memory. The second comparison isolates what is new. It tests whether the spiking-specific price changes which layers get budget, and whether that change matters for accuracy.
What Prior Work Does and Leaves Open
Three conversion papers supply the mechanism we build on, and all three fix the time budget globally. Fast-SNN proves the quantized-ANN/SNN equivalence and reaches 71.31% ImageNet top-1 with VGG-16 at $T=3$. It needs signed integrate-and-fire neurons that emit negative spikes to cancel premature firing. Efficient ANN-SNN Conversion with Error Compensation Learning splits conversion error into clipping, quantization and uneven-arrival components. It learns clipping thresholds, uses dual-threshold neurons that also fire negative spikes, and initializes membranes at half threshold; it reports 94.75% on CIFAR-10 at $T=2$ and 74.15% on ImageNet at $T=16$. Activation-Aware Redistribution shows that channel-wise distribution mismatch drives conversion error and fixes it with per-channel thresholds and offsets, at a global $T=64$ for ViT-L. We reuse their threshold machinery. Our change is the choice of how many levels each layer gets, which none of them treat as a variable.
The sensitivity literature supplies the allocation signal but was built for a different cost. HAWQ ranks layers by the top Hessian eigenvalue of the loss with respect to weights and assigns bits accordingly. Channel-Wise Mixed-Precision Quantization and Beyond Outliers refine the granularity and remove the need for calibration data. All of them measure weight sensitivity and price precision in bits. In a converted network, the quantity that becomes time steps is the activation level count, so we compute curvature with respect to activations and price it in spikes.
Hardware-aware search supplies the cost-modeling method, and its spiking variants motivate the gap. Balaskas et al. search mixed precision and pruning with lookup-table energy models of the Eyeriss accelerator, but only for ordinary networks. SpikeNAS, HASNAS and SNASNet search spiking architectures for direct training and apply uniform precision, for example 8-bit fitness evaluation in HASNAS. HASNAS states that converted networks are less efficient than directly designed ones on in-memory-compute hardware. We take this as a claim to test, not to ignore. Its comparison used uniform-budget conversions, and our hypothesis is that uniform budgets are the inefficiency. Finally, PTQ4SNN shows that membrane potentials, the running sums each neuron stores, dominate memory traffic, and it quantizes them with mixed 2/4/8-bit allocation on directly trained networks. Our cost model inherits that lesson and charges for membrane bits, which conversion work currently ignores.
Research Program
Execution order that makes heterogeneous budgets exact. Conversion methods usually run all layers in lockstep, so a downstream neuron starts firing before it has received all its input. That premature firing is the "uneven arrival" error, and it is what negative spikes exist to cancel. We instead run layer-sequentially. Layer $l$ first integrates every spike from layer $l-1$. It then fires for $L_l$ steps through a unipolar integrate-and-fire neuron with threshold $\theta_l$ and initial membrane $\theta_l/2$. Under this schedule the spike count equals a round-to-nearest quantization of the clipped activation into $L_l$ levels, exactly, with no negative spikes. Each layer's window can have a different length, because layers never overlap in time.
The price is latency. Total latency becomes $\sum_l L_l$ rather than roughly $T$ plus depth, and we will report it rather than hide it. We also run a lockstep variant with heterogeneous budgets to measure how much accuracy the exact schedule buys and how much latency it costs.
Sensitivity of activation levels. For layer $l$ with clipping threshold $\theta_l$, the rounding step is $\Delta_l = \theta_l / L_l$. Under the standard model of rounding error as uniform noise, the expected loss increase is approximately
$$\Omega_l(L_l) \approx \frac{\operatorname{tr}(H_l)}{2} \cdot \frac{\Delta_l^2}{12},$$
where $H_l$ is the Hessian of the training loss with respect to layer $l$'s activations. Its trace is estimated with Hutchinson's random-probe method on a few hundred calibration images. The $\Delta^2/12$ factor is the variance of uniform rounding noise. We first check empirically that this ranking predicts which layers actually lose the most accuracy when their budgets are cut. If it does not, the allocator falls back to measured per-layer accuracy drops.
A spiking cost model. The hardware cost of layer $l$ with $N_l$ neurons is
$$C_l(L_l) = e_{\text{ac}}\, \bar{r}_l(L_l)\, N_l\, F_l \;+\; e_{\text{mem}}\, N_l\, m_l,$$
where $\bar{r}_l$ is the measured mean spike count per neuron (bounded by $L_l$) and $F_l$ is the fan-out. The constant $e_{\text{ac}}$ is the energy of one accumulate. The term $e_{\text{mem}}$ is the per-bit membrane storage and access cost, and $m_l$ is the membrane bit-width needed to hold the accumulated input without overflow. The bit-width $m_l$ grows with $\log_2(F_l^{\text{in}} L_{l-1})$, so budget upstream raises memory downstream. Following Balaskas et al. and PTQ4SNN, the constants come from a lookup table for a simulated processing-element array. We will not claim physical-chip measurements, because the file gives us none.
Allocation and a narrow architecture search. We minimize $\sum_l \Omega_l$ subject to $\sum_l C_l \le B$ for a range of budgets $B$. The coupling between layers is chain-structured, because $m_l$ depends on $L_{l-1}$. Dynamic programming over $L_l \in \{1, 2, 3, 5, 7, 15\}$ therefore solves this exactly, in the same spirit as the cell-by-cell dynamic program of SpikeNAS. The hardware-aware search component is deliberately narrow. Spike cost is roughly width times levels, so we jointly search per-stage channel width and level count. The question is whether, at fixed cost, a converted network prefers wide layers with few levels or narrow layers with many. The source network is then fine-tuned with quantization-aware training at the chosen per-layer levels, using the learnable clipping of Liu et al., and converted.
Baselines
The naive reference point is uniform-budget conversion with unipolar neurons, run with our layer-sequential schedule. A single $L$ is applied to every layer and swept so that its cost matches ours at each budget $B$. This isolates allocation from execution order.
The strongest existing low-latency methods are Fast-SNN (71.31% ImageNet top-1, VGG-16, $T=3$) and Liu et al. (94.75% CIFAR-10, ResNet-18, $T=2$; 74.15% ImageNet, VGG-16, $T=16$; 0.008 mJ on CIFAR-100 VGG-16 at $T=4$). Both use negative spikes. We report their published numbers and rerun them on the same source networks, data and fine-tuning epochs. We compare at matched synaptic operations and membrane memory, not matched $T$, because a matched $T$ hides the cost differences. Beating them means one of two outcomes. Either we match their accuracy within 0.5 points at equal or lower modeled energy while using only unipolar spikes, or we exceed them at equal energy. We also report their accuracy when their neurons are restricted to unipolar firing, which is the fair hardware-compatible comparison.
The ablation that isolates our key ingredient is HAWQ-style allocation priced in bits. It uses the same activation Hessians, the same dynamic program, the same fine-tuning and the same execution schedule, and changes only the cost function to the ordinary bit-memory cost. If this ablation matches our accuracy at equal spike cost, the spiking cost model adds nothing.
A second ablation replaces the Hessian sensitivity with measured per-layer accuracy drops, to test whether curvature is needed at all. For the membrane-memory axis, we compare against applying PTQ4SNN-style post-hoc mixed membrane quantization to a uniform conversion. For reference, PTQ4SNN reports 75.16% on SDT-8-768 at W4/M4, but on directly trained networks, so that figure is context, not a head-to-head number. All methods receive identical fine-tuning epochs and an identical calibration set.
Why not just…?
Why not just use uniform low-bit quantization-aware training and convert, as Fast-SNN does? This is the uniform baseline above, so the experiments answer it directly. Our prior expectation comes from the consistent advantage of mixed over uniform precision in ordinary networks (HAWQ; Balaskas et al.), and from the fact that Fast-SNN needed negative-spike neurons to stay accurate at $T=3$. It is possible that conversion with half-threshold initialization is already robust enough that heterogeneity gains little on small CIFAR networks. If the gain is under half a point there, we will say so and concentrate on ImageNet and deeper networks, where sensitivity spreads are larger.
Why not just run an existing mixed-precision allocator such as HAWQ on the source network and convert? This is the real threat and the reason for the bit-cost ablation. If spike counts track bits closely enough, the cheaper existing tool suffices. We expect divergence for two reasons. Spike cost is linear in levels while bit cost is logarithmic, and spike cost is weighted by fan-out, which penalizes budget in early, high-resolution layers far more than bit memory does. The experiment reports both the accuracy gap and how differently the two allocators distribute levels across depth. A small allocation difference with no accuracy gap would falsify the novelty.
Why not just learn per-layer levels by gradient descent with a spike-count penalty, skipping Hessians and search? Learnable clipping already exists (Liu et al.), and adding a differentiable level count with a spike penalty is a plausible, simpler allocator. We include it as a third allocator under the same cost model. The claim we stake is on per-layer spike budgets priced in spikes and executed exactly with unipolar neurons, not on Hessians specifically. If the learned allocator matches the dynamic program, we will recommend it as the practical method and keep the Hessian analysis only as an explanation of which layers receive budget.
Why not just train the spiking network directly and search its architecture, as SpikeNAS and SNASNet do? Direct training is reported to take up to ten times longer than ordinary training (SNASNet), while conversion scales to ViT-L in about 150 seconds of calibration (Cao et al.). On CIFAR-10 at small scale, though, direct search is competitive: SNASNet reports 93.73% at 5 time steps and SpikeNAS about 92% with 1.04M parameters. We will compare at matched modeled energy on CIFAR-10 and CIFAR-100, and we expect at most parity there; the advantage of conversion is meant to appear at ImageNet scale.
Proposed Experiments
We use VGG-16 and ResNet-18/20 on CIFAR-10, CIFAR-100 and ImageNet-1K. A ViT experiment on ImageNet is deferred until the convolutional results are in, because the attention operators need the separate machinery of Towards High-performance Spiking Transformers. For each method we trace accuracy against modeled energy across five cost budgets, and we report synaptic operations, membrane memory, latency in time steps for both schedules, and the per-layer level allocation.
The first decisive check is predictive. On ResNet-18/CIFAR-100, the activation-Hessian ranking must correlate with measured per-layer accuracy loss under budget cuts, with a Spearman correlation above 0.6. If it does not, the curvature signal is dropped.
The second check is the main result. On ImageNet with VGG-16, the spike-priced allocation must beat uniform budgets by at least one top-1 point at matched modeled energy. It must also beat the bit-priced ablation by a margin larger than the seed-to-seed spread over three seeds.
The third check addresses the hardware argument. With unipolar spikes only, our method must come within one point of the published accuracy of the negative-spike methods at equal or lower modeled energy. We will also report the latency cost of the layer-sequential schedule, so readers can judge whether exactness is worth it.
The width-versus-levels search is a secondary analysis. Its outcome, whichever direction it goes, tells future spiking-oriented architecture searches whether converted networks should be designed wider or with finer per-layer resolution.
References
- Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN
- HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision
- Efficient ANN-SNN Conversion with Error Compensation Learning
- Towards Training-Free and Accurate ANN-to-SNN Conversion via Activation-Aware Redistribution
- Channel-Wise Mixed-Precision Quantization for Large Language Models
- Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity
- Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization
- SpikeNAS: A Fast Memory-Aware Neural Architecture Search Framework for Spiking Neural Network-Based Embedded AI Systems
- HASNAS: A Hardware-Aware Spiking Neural Architecture Search Framework for Neuromorphic Compute-in-Memory Systems
- Neural Architecture Search for Spiking Neural Networks
- PTQ4SNN: Membrane-Aware Post-Training Quantization for Spiking Neural Networks
- Towards High-performance Spiking Transformers from ANN to SNN Conversion