A spiking neural network computes with brief all-or-nothing pulses spread over several time steps, and on brain-inspired chips it spends energy only when a pulse fires. Two separate lines of work have tried to make these networks cheaper. One stops computation early for inputs that are easy to classify. The other trains the network to fire fewer pulses at every step. Both treat their target as a separate quantity. The true bill, however, is a single number: how many additions the chip performs for one input, summed over all the time steps that input actually used. We propose to train spiking networks directly against the expected value of that bill, using a stopping rule that is itself built from spikes.
The two levers interact, and that is why a joint objective matters. A firing-rate penalty applied uniformly over time suppresses exactly the early activity a network needs in order to stop early. A stopping rule trained on a network that was never asked to decide early has little early evidence to work with. If each step's spike cost is weighted by the probability that the input is still being processed, the network is pushed to concentrate its spikes in the first few steps for easy inputs and then stop. Our claim is that this front-loading cannot be obtained by combining existing methods after the fact. It should yield a strictly better curve of accuracy against operation count, it should keep spikes purely binary, and it should avoid the training collapse that plain activity penalties cause.
Core Claim
SEENN motivates the proposal. It observes that a spiking classifier that is correct at time step $t$ almost always stays correct afterward, and it exploits this with a hand-tuned confidence threshold (SEENN-I) or with a separate non-spiking policy network trained by reinforcement learning (SEENN-II). Activity Pruning for Efficient Spiking Neural Networks constrains the design. It proves that adding a firing-rate penalty to the loss drives gradient norms to zero, which traps networks in silent states. Its fix, the adaptive-threshold neuron AT-LIF, scales each emitted spike by a learned threshold, so spikes are no longer binary. Spikingformer shows why this matters: non-binary spike values introduce integer-float multiplications and break pure event-driven execution. No paper in this file optimizes stopping and per-step activity under one cost. Methods that learn thresholds against a differentiable compute penalty, such as Adaptive Sparse ViT for token dropping in vision transformers, have not been carried over to temporal stopping inside a spiking network.
The mechanism works as follows. The output layer integrates its inputs without leak, so its membrane potentials act as running class scores. Let $m_t$ be the gap between the top two scores at step $t$. This gap can be computed with additions and one comparison, so the stopping signal needs no dense auxiliary network. During training we soften the decision to $h_t = \sigma((m_t - \tau)/s)$, where $\tau$ is a learned stopping threshold and $s$ a temperature that is annealed toward a hard step. The probability of still running at step $t$ is $S_t = \prod_{k<t}(1-h_k)$. Writing $C_t$ for the synaptic operations performed at step $t$ (spikes times fan-out, counted exactly in simulation) and $\ell_t$ for the cross-entropy of the readout at step $t$, we minimize
$$L = \sum_{t=1}^{T} S_t\, h_t\, \ell_t \;+\; \lambda \sum_{t=1}^{T} S_t\, C_t ,$$
with $h_T = 1$, so that every sample stops by the final step $T$. The first term is the expected classification loss at the step where the sample stops. The second is the expected operation count for that sample. This form follows the halting-distribution construction used for adaptive computation in recurrent networks, and the IDK Cascades idea of folding a $\lambda \cdot C$ cost into the risk it minimizes. What is new here is that the cost is the spike count of a binary network, weighted by the probability of still running. At test time the rule is hard: stop the first time $m_t > \tau$. Sweeping $\lambda$ traces the trade-off curve, and we retrain at each $\lambda$.
We expect the cost weighting to avoid AT-LIF's collapse for a specific reason. Under a uniform rate penalty, the only way to reduce cost is to fire less, and firing less starves the backward pass. Under the survival-weighted cost, the optimizer has a second route: raise early score gaps, stop sooner, and remove all later cost at once. That route rewards strong early spiking rather than silence. This is a hypothesis, and we test it by measuring gradients rather than by assuming it.
Research Program
We first build the objective on SEW-ResNet18 and ResNet-19 backbones with ordinary binary leaky integrate-and-fire neurons and surrogate gradients. The neurons have no threshold scaling. We then port it to Spikingformer using the TheBrainLab/Spikingformer codebase, whose residual design already keeps execution purely event-driven. Its energy tooling converts operation counts to energy with a fixed per-addition constant. We report raw operation counts as the primary metric and energy only as a derived quantity, because the file's hardware evidence (Energy-efficient neuromorphic classifiers) says chip savings depend on sparse binary traffic, not on operation counts alone.
The analysis has two parts. First, we plot spike count against time step, split by the step at which each sample stopped. If the claim holds, early-stopping samples should show spike mass shifted toward early steps relative to a network trained with the per-step loss of Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks. Second, we track per-layer gradient norms and the fraction of silent neurons across a range of $\lambda$ values, side by side with a plain rate-penalty run. This is the empirical counterpart to AT-LIF's Theorem 1.
Event-camera data needs separate treatment, because information there arrives over time and stopping means no longer reading the stream. Dynamic Spatio-Temporal Pruning reports that cutting time steps on DVS128 Gesture even improved accuracy by 0.69%. On these datasets we therefore ask whether the learned rule recovers that effect without the label-dependent KL stopping criterion that paper uses.
Baselines
The naive reference is a fixed-$T$ spiking network trained with the per-step loss from Rethinking the Membrane Dynamics, evaluated at every $T$ from 1 to the training $T$. This gives a curve of accuracy against operations with no adaptivity at all.
The strongest competitor is SEENN-I applied to that same network: the threshold $\alpha$ on maximum softmax probability is swept post hoc. It is the strongest because it is nearly free and already reports 96.07% CIFAR-10 accuracy at an average of 1.09 time steps, against 6-step baselines. On ImageNet it reports 67.99% at 2.35 average steps against the standard 4 (SEENN). We use the same backbone, data, augmentation, and training epochs as for our method, and give SEENN-I the same number of threshold settings as we have $\lambda$ values. Beating it means a curve of accuracy against operations per sample that lies above SEENN-I's over a range of budgets on at least CIFAR-10 and CIFAR10-DVS. Matching it on the time-step axis alone does not count.
The second strong baseline is the sequential composition of AT-LIF training followed by SEENN-I stopping. AT-LIF alone reports an 88.6% reduction in synaptic operations on CIFAR-10 against a vanilla network, with negligible accuracy loss (Activity Pruning). Composed with stopping, it is the obvious "do both separately" answer. We run it with AT-LIF's thresholds absorbed into the weights after training, so the deployed network is binary, and we report whether the absorption costs accuracy.
SEENN-II is included where compute allows, with the operations of its dense policy network counted in its cost.
The key ablation keeps our stopping rule and expected classification loss but replaces the survival-weighted cost $S_t C_t$ with an unweighted $C_t$. This isolates whether weighting by survival, and not merely training the stopping rule end to end, produces the gain. On ImageNet with Spikingformer, the reference point is Spikingformer-8-512 at 74.79% top-1 and 7.46 mJ with $T=4$ (Spikingformer).
Why not just…?
Why not just threshold confidence on a well-trained network, as SEENN-I does? This is the real threat. The per-step training loss of Rethinking the Membrane Dynamics already makes early readouts usable, and SEENN-I's 1.09-step CIFAR-10 result leaves little room on the time-step axis. Our bet is on the other axis: operations per step. A network that was never charged for spikes fires densely even at step one. If our curve does not clear SEENN-I's curve on the same backbone, the claim is false and we will report it as such. In that case the remaining contribution would be the analysis of where spikes fall in time.
Why not just train with AT-LIF and then add early stopping? The sequential pipeline has two problems. AT-LIF's penalty is uniform over time, so it suppresses early spikes as much as late ones, which works against early stopping. Its stability also depends on non-binary scaled spikes, which Spikingformer shows break event-driven execution. The composed baseline and the unweighted-cost ablation together show whether joint weighting adds anything beyond stacking the two methods.
Why not just use a conventional network with early-exit cascades? IDK Cascades reports a 37% compute reduction on ImageNet, but every remaining operation is still a multiply-accumulate. Our question concerns addition-only hardware, where the relevant cost is spike traffic. We report the cascade's FLOPs alongside our operation counts for transparency, but we do not claim that operation counts on different hardware are directly comparable without chip measurements.
Proposed Experiments
The decisive experiment is the CIFAR-10 and CIFAR10-DVS comparison of trade-off curves (accuracy against measured synaptic operations per sample) for our method, SEENN-I, AT-LIF followed by SEENN-I, and the unweighted ablation. All use identical backbones and training budgets and three seeds each.
A second check reproduces AT-LIF's collapse with a plain rate penalty and tests whether our objective trains stably at matched or lower operation counts. The evidence is gradient norms and silent-neuron fractions, using binary spikes throughout.
The third check runs DVS128 Gesture, where Spikingformer reports 98.6% accuracy and Dynamic Spatio-Temporal Pruning reports a 50% reduction in time redundancy. It tests whether the learned rule matches that time saving without label-based stopping.
ImageNet with SEW-ResNet18 and Spikingformer-8-512 tests whether the effect holds at scale. If a Loihi-class chip is available, we will measure energy for the best CIFAR10-DVS models. If not, the results remain claims about operation counts, which is the same limitation AT-LIF and SEENN acknowledge.
References
- SEENN: Towards Temporal Spiking Early Exit Neural Networks
- Activity Pruning for Efficient Spiking Neural Networks
- Spikingformer: A Key Foundation Model for Spiking Neural Networks
- Adaptive Sparse ViT: Towards Learnable Adaptive Token Pruning by Fully Exploiting Self-Attention
- IDK Cascades: Fast Deep Learning by Learning not to Overthink
- Energy-Efficient Neuromorphic Classifiers
- Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks
- Dynamic spatio-temporal pruning for efficient spiking neural networks