Spiking neural networks are networks of model neurons that communicate with brief all-or-nothing pulses, and they underpin large brain simulations and low-energy hardware. They are almost always trained and run on one fixed simulation clock, typically 1 ms or 0.1 ms per step. Change that clock after training and the network usually stops working. Two things break when the clock changes. First, the decay factors that were baked in for the old step no longer match. Second, and more subtly, every spike is snapped to the step grid, and the reset that follows it lands at the wrong moment. Linear state space models (sequence models that learn a continuous-time linear system and then discretize it with a step size $\Delta$) solved the analogous problem for non-spiking networks: rescale $\Delta$ at test time and the same weights work at a new rate. Spiking neurons are not linear, though. They have thresholds, hard resets, and in realistic reduced models an exponential spike upstroke and a slow adaptation current.
This note proposes a spiking neuron layer whose one-step update is built to approximate the true continuous-time flow of the neuron between spikes. The update locates each threshold crossing inside the step and applies the reset at that sub-step time. It also defines its training gradients in continuous time rather than per bin. The claim is that such a network, trained at a single step size, keeps its accuracy and its spike timing when deployed at step sizes 4 to 8 times finer or coarser, and under irregular input timing. Networks that only rescale their decay constants will not. If the claim holds, one set of learned parameters could be trained at a coarse, cheap step and then checked at a fine one. This matters for connectome-scale simulation and for neuron models fit to physiological recordings.
Core Claim
Gap 1 in the source material is plain: no work demonstrates zero-shot invariance to the step size in a spiking model. Structured State Space Model Dynamics and Parametrization for Spiking Neural Networks (SiLIF) is the closest existing work. It borrows the state space model machinery of a learnable per-neuron timestep, log-domain decay parametrization, and complex-valued resonant states, and it reaches 82.03% on Spiking Speech Commands and 95.88% on Google Speech Commands with delays. Its $\Delta t$, however, is a static trained parameter, and it is never varied at inference. A Spiking Heterogeneous Harmonic Resonate-and-Fire State Space Model for Time Series stabilizes long-horizon discretization with implicit and implicit-explicit schemes. It does so by replacing the hard reset with a smooth one, and it too never tests transfer across rates. The two papers also disagree about resets. The resonate-and-fire paper drops the hard reset to gain parallel training, while SiLIF argues the reset is the nonlinearity that makes spiking models beat non-spiking ones. Our design keeps the hard reset and makes it step-consistent. That lets us test directly whether reset-bearing models can also be rate-invariant.
The specific new ingredient is a sub-step reset map. For a neuron with membrane voltage $V$ and adaptation current $w$, suppose the voltage crosses threshold $\theta$ at an estimated fraction $s\in(0,1)$ of a step of length $\Delta$. The update integrates the continuous dynamics to the crossing, applies the reset, and then integrates the remaining $(1-s)\Delta$ from the reset state:
$$ (V,w)_{t+\Delta} = \Phi_{(1-s)\Delta}\big(V_r,\; w_{t+s\Delta}+b\big), \qquad (V,w)_{t+s\Delta} = \Phi_{s\Delta}\big((V,w)_t\big). $$
Here $\Phi_\tau$ is the flow of the neuron equations over time $\tau$. $V_r$ is the reset voltage and $b$ is the adaptation jump, both from the adaptive exponential integrate-and-fire (AdEx) model of Firing patterns in the adaptive exponential integrate-and-fire model. The emitted spike carries the continuous time $t+s\Delta$, not the bin index. For the linear subthreshold part, $\Phi$ is the exact matrix exponential; this is the same zero-order-hold discretization used in Efficiently Modeling Long Sequences with Structured State Spaces. The exponential spike-initiation term is handled by an exponential-Euler step around the current voltage, the scheme Integration of Continuous-Time Dynamics in a Spiking Neural Network Simulator uses in NEST. The crossing fraction $s$ comes from a single Newton or secant correction on the within-step voltage trajectory. The surrogate gradient, which is the smooth stand-in for the spike's zero derivative, is defined as a function of voltage slope and crossing time rather than of per-bin voltage. Its effective width in milliseconds therefore does not shrink or grow with $\Delta$.
Research Program
The layer is built in two neuron families. The first is a SiLIF-style adaptive leaky neuron with real and complex (resonant) states, which allows a like-for-like comparison with Structured State Space Model Dynamics and Parametrization for Spiking Neural Networks. The second is AdEx, including its bursting and adaptation regimes, which tests the nonlinear-plus-reset case named in Gap 2.
All parameters are stored as continuous time constants, never as per-step decay factors. The input side is treated the same way. Input spikes keep their continuous timestamps where the dataset provides them, and synaptic delays are stored in milliseconds.
The second strand makes the online-learning machinery rate-aware. BrainTrace, from Model-agnostic linear-memory online learning in spiking neural networks, trains a 125,000-neuron Drosophila connectome model in 8.9 GB of GPU memory using eligibility traces, which are running per-synapse records of recent activity that approximate the gradient. Its trace decays are hardcoded as $\exp(-dt/\tau)$ for a fixed $dt$. We will give the traces the same per-step $\Delta$ as the neurons. We will also add the crossing-time correction to the trace update, following the per-step modulation of $\Delta$ in Simplified State Space Layers for Sequence Modeling. The test is whether the approximations in pre-post eligibility propagation (pp-prop) still hold when $\Delta$ varies from step to step.
The third strand is evaluation that does not itself depend on a grid. Spike-train agreement across step sizes is measured with the van Rossum distance, computed in linear time per train with the method of On the efficient calculation of van Rossum distances. It is also measured with the ISI-distance of Measuring spike train synchrony between neuronal populations, which has no time-scale parameter to tune. Both operate on continuous spike times, so a fine-step and a coarse-step run can be compared directly.
Baselines
The naive reference point is a standard discrete adaptive leaky integrate-and-fire network. Its decay factors are learned directly and it is simply run at the new step size. This number shows how badly rate shifts hurt by default.
The strongest existing method is SiLIF and C-SiLIF from Structured State Space Model Dynamics and Parametrization for Spiking Neural Networks. The reported in-distribution numbers are 82.03% on Spiking Speech Commands and 95.88% on Google Speech Commands with delays. SiLIF already has continuous-time parameters, so it is the fairest competitor. We will run it in its most rate-friendly form: its learned time constants are converted to decays for the new $\Delta$ at test time, and its delays are rescaled from steps to milliseconds. SiLIF and our model use the same architecture, width, data, optimizer, and hyperparameter search budget, and they are trained at the same $\Delta_{\text{train}}$. Our model counts as beating it only under three conditions. First, it must match its in-distribution accuracy within one point. Second, it must retain accuracy at shifted step sizes where SiLIF measurably degrades. Third, its spike-time distance to a fine-step reference must be lower than SiLIF's.
On Spiking Heidelberg Digits, BrainTrace's reported 95.33% is the in-distribution bar for the online-learning strand.
The key ablation removes only the sub-step reset. It keeps exact subthreshold integration and continuous parameters but snaps crossings to the step boundary. A second ablation keeps the sub-step reset but uses the per-bin surrogate gradient. Together these separate the contribution of reset timing from that of gradient definition.
For the non-spiking reference, S4D is run by rescaling $\Delta$. It bounds what a linear model achieves on the same task. No reported result was found on Lacuna for S4D under step-size shift on these datasets.
Why not just…?
Why not just recompute the decay factors from learned time constants at test time? This is the obvious fix, and it is the SiLIF baseline above. It is also a real threat. For fine test steps it may nearly suffice, because spike-snapping error shrinks with $\Delta$. It cannot fix the coarse-step case, where several threshold crossings or a crossing plus a full reset fall inside one bin. It also cannot fix networks that learned to depend on the one-spike-per-bin structure of the training clock. The sweep over test step sizes settles the question. If recomputed decays hold accuracy within one point at 4 to 8 times coarser steps, our sub-step reset adds nothing for classification. In that case the contribution narrows to spike-timing fidelity and simulation cost, and we will report it that way.
Why not just train with randomly sampled step sizes? Step-size augmentation is cheap and may give robustness inside the sampled range. It gives no guarantee outside that range, and it spends model capacity on memorizing several discretizations rather than one continuous system. We will train the SiLIF baseline with augmentation over half the test range and evaluate on the other half. We will also combine augmentation with our layer to see whether the two gains add.
Why not just resample inputs to the training rate? On event datasets there is nothing to interpolate: rebinning the spikes is the rate change. For physiological fitting, Integrate-and-Fire models with adaptation are good enough shows that 2 ms spike-timing prediction depends on the neuron's own integration dynamics. Resampling the input leaves the neuron's internal discretization error untouched. Resampling to the training rate will nonetheless be included as a baseline in the irregular-timing experiment.
Why not just use an adaptive Runge-Kutta solver? Adaptive solvers work for single neurons, and a fine-step Runge-Kutta run will serve as our reference trajectory. At scale they force fine-grained synchronization across processes, which Integration of Continuous-Time Dynamics in a Spiking Neural Network Simulator shows is a bottleneck: its waveform relaxation gives up to a threefold speedup. Adaptive solvers also cache substeps for backpropagation, which BrainTrace shows exceeds 32 GB on the connectome model.
Proposed Experiments
The classification experiments use Spiking Heidelberg Digits and Spiking Speech Commands from The Heidelberg spiking datasets, whose spikes carry continuous timestamps, plus Google Speech Commands. Models are trained at a single step of 1, 2, or 4 ms. Each is tested at 0.25, 0.5, 1, 2, 4, and 8 ms, and under jittered, irregular steps. The headline metric is the accuracy gap between the shifted and the training step. It is reported alongside synaptic operations, the event-count measure of compute used in the SiLIF paper.
The physiological experiment fits AdEx neurons, in the manner of An analytical approximation to the AdEx neuron model allows fast fitting to physiological data. The fitting targets are recordings of layer 5 pyramidal cells driven by fluctuating current injection, as in Integrate-and-Fire models with adaptation are good enough, if such recordings are accessible. The fallback target is a detailed two-compartment model from A Systematic Study of the Input/Output Properties of a 2 Compartment Model Neuron With Active Membranes. Each model is fit at 0.1 ms and evaluated at 0.5 to 1 ms. The metric is the coincidence factor, which measures how many spikes fall within 2 ms of the real ones. The reported bar is about 65% of the neuron's intrinsic reliability. The decisive result is that our layer retains that level at 1 ms while snapped-reset AdEx does not. We will also check that all eight AdEx firing patterns survive the coarser step.
The scale experiment runs the Drosophila connectome model with rate-aware BrainTrace at 0.1 ms and at coarser steps. It measures ISI-distance and van Rossum distance of the population spike trains to the fine-step run, together with wall-clock time and memory. The question is how coarse the step can go before network activity diverges.
The single most decisive check is the coarse-step comparison against SiLIF with recomputed decays. If the sub-step reset does not open a clear gap there, the core claim is falsified.
References
- Structured State Space Model Dynamics and Parametrization for Spiking Neural Networks
- A Spiking Heterogeneous Harmonic Resonate-and-Fire State Space Model for Time Series
- Firing patterns in the adaptive exponential integrate-and-fire model
- Efficiently Modeling Long Sequences with Structured State Spaces
- Integration of Continuous-Time Dynamics in a Spiking Neural Network Simulator
- Model-agnostic linear-memory online learning in spiking neural networks
- Simplified State Space Layers for Sequence Modeling
- On the efficient calculation of van Rossum distances
- Measuring spike train synchrony between neuronal populations
- Integrate-and-Fire models with adaptation are good enough
- The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks
- An analytical approximation to the AdEx neuron model allows fast fitting to physiological data
- A Systematic Study of the Input/Output Properties of a 2 Compartment Model Neuron With Active Membranes