Neuromorphic chips run spiking neural networks, which are networks whose neurons emit discrete pulses once an internal voltage crosses a firing threshold. On these chips, and on other physical accelerators, the synaptic weights are often fixed after fabrication or deployment. When input statistics shift or the chip develops defects, the only practical adjustments left are small per-neuron quantities such as firing thresholds. Existing ways to set them either need a backward pass that forward-only hardware cannot run, or need the model to have been pre-trained with special normalization layers. A search that uses only forward evaluations would avoid both requirements. Such a search tries many candidate threshold vectors, scores each by running the network, and keeps the better ones. The obstacle is that this search space is mostly flat. Classification decisions depend on integer spike counts, silent or saturated neurons do not react to small threshold changes, and faults push neurons into regimes where nothing responds. Most candidates therefore score exactly the same.
We propose to make forward-only threshold search on frozen spiking hardware practical by combining two pieces. The first is a GPU simulator that evaluates very large populations of threshold candidates cheaply. It does this by exploiting the fact that every candidate shares the same frozen weights and synapse structure. The second is a search rule built for plateaus. It breaks ties using a graded, forward-observable signal: how far each neuron's voltage sits from the band where it responds to input. It also stops the search's step size from shrinking when progress stalls only because the landscape is flat. The claim is that on frozen networks with realistic faults, this recovers accuracy that gradient-based threshold tuning cannot. Gradient methods fail here because their gradients vanish on exactly these plateaus.
Core Claim
Threshold adaptation of a frozen spiking network is a flat-landscape black-box problem, not a gradient problem. It becomes tractable when two conditions hold. Evaluation must be cheap enough that population sizes in the thousands cost close to one forward pass of memory traffic. The search must also be able to tell apart candidates that tie on the task objective. Neither condition is met by current work. Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks adjusts firing thresholds after deployment in two ways. TM-NORM needs pre-training with membrane-potential batch normalization, and TM-ENT needs a backward pass for entropy minimization. The authors flag both requirements as limits. Practical Mechanism for Fault-Tolerant Spiking Neural Networks via Simple Input Control Based on Learnable Fragmentation shows why gradients fail under faults. Stuck-at defects push membrane potentials out of the narrow "surrogate gradient corridor," the voltage range where the approximate derivative used to train spiking networks is nonzero, and learning stalls. That paper states that its fix is trained offline and that online, on-chip adaptation is missing. Our proposal targets that gap with a method that uses no gradients and no special pre-training.
Research Program
The first component is a population-aware spiking simulator. General libraries such as evosax: JAX-based Evolution Strategies keep the whole ask-evaluate-tell loop on the accelerator by vectorizing over candidates. In that loop, ask samples candidates, evaluate scores them, and tell updates the search distribution. For dense layers, vectorizing over candidates reduces to an ordinary batched matrix multiply with shared weights, which is already efficient. We will not claim a gain there.
The gain we expect is in event-driven sparse simulation, where only the synapses of neurons that actually spiked are read. Brian2CUDA shows that this propagation step causes scattered memory reads. It fixes them by bundling synapses with equal delays and routing spikes through circular queues. With many candidates, the scatter gets worse, because different candidates fire different neurons.
We will reorganize the simulator around a population-minor layout. Each neuron's state for all candidates is stored contiguously, so a warp covers 32 candidates of the same neuron. A warp is the group of 32 GPU threads that execute together. Spike events from all candidates are then grouped by the neuron that sent them, so each frozen weight row is fetched once and applied to every candidate that fired it. This is the same move as the columnar layouts in Column-Oriented Datalog on the GPU. Neighboring threads touch neighboring addresses, so each 128-byte memory transaction is fully used. Here the shared axis is "same neuron across candidates" rather than "same attribute across tuples."
Faults are modeled as stuck-at weights that are shared across all candidates. The digital model of the chip therefore stays a single fixed network. The Brian2CUDA paper also warns that per-phase kernel launches hurt small networks. For that reason we fuse the state update, threshold check, and reset into one kernel.
The second component is plateau-aware search. We start from a separable evolution strategy over log-thresholds, run inside the same device-resident loop. We change it in three ways.
First, fitness is lexicographic. The primary score is the unsupervised test-time objective: prediction entropy, or agreement across augmentations, on the streaming batch. The secondary score is graded and visible without a backward pass. It is the mean distance of each neuron's membrane potential from its responsive band, penalizing both silence and saturation. This plays the role that rectified cost functions play in Generating gradients in the energy landscape using rectified linear type cost functions. It supplies direction inside plateaus. Because it is used only to break ties, it does not reshape the landscape, which avoids the over-smoothing failure reported in Wang et al..
Second, a plateau detector measures the fraction of the population tied on the primary score. When that fraction is high, step-size decay is frozen. This addresses the collapse described by Cauwet.
Third, while on a detected plateau, the search keeps a persistent drift direction instead of redrawing it each generation. This borrows the idea behind the two-state Markov-chain operator selection in Speeding Up Hyper-Heuristics With Markov-Chain Operator Selection and the Only-Worsening Acceptance Operator. That work proves fast plateau crossing on discrete Jump functions and leaves continuous, high-dimensional plateaus open. We carry its mechanism into that open setting as an empirical question, not as an inherited guarantee.
Two limits on scope follow. First, candidates are scored in simulation, so what we test is whether the thresholds found transfer to a network with the same faults. Second, the search needs only forward passes and scalar comparisons, which is the property that would let it run on the chip itself later.
Baselines
The naive reference point is doing nothing: the frozen faulty network under shift, with its original thresholds. A second naive reference is plain random search over thresholds at matched evaluation budget. It tells us whether any structured search helps on a landscape this flat.
The strongest existing methods are TM-NORM and TM-ENT from Threshold Modulation. We will run them on the same pre-trained models, the same shifts, and the same streaming batches. For TM-NORM we will pre-train a matched model with membrane-potential batch normalization, so that it is compared on its home ground as well as on standard models where it cannot run as designed. TM-ENT gets its gradient access in simulation. The file reports no numeric results for these methods: no reported result found on Lacuna for TM-NORM or TM-ENT accuracy under any named shift benchmark. We will therefore reproduce them and report our reproduced numbers as the bar. Beating them means that under injected stuck-at faults, our method recovers more accuracy than TM-ENT at equal wall-clock time, while needing no backward pass. It also means we are within noise of TM-NORM on models pre-trained with its normalization, and clearly ahead of it on models without.
The strongest general black-box competitor is an off-the-shelf evolution strategy from evosax: separable CMA-ES and SNES run on the standard vectorized simulator. It gets the same population sizes, number of evaluations, and hyperparameter-tuning budget as ours.
Three ablations isolate the ingredients. The first removes the secondary score. The second removes step-size freezing and persistent drift. The third keeps the search but swaps the population-minor layout for a candidate-major one, which isolates the systems contribution.
On the systems side, we will profile each layout for bytes read per spike event and sector-per-request efficiency. Beating the standard layout means throughput, in candidate-samples per second, that scales sublinearly in population size on the sparse event path.
Why not just…?
Why not just backpropagate through the GPU simulation with surrogate gradients and copy the thresholds to the chip? This is the most serious alternative, because the simulation is differentiable. Its weakness is predicted by the fragmentation paper: faulted neurons leave the surrogate gradient corridor, and their threshold gradients go to zero. We will run it as a baseline and sweep fault density. If it matches our method at every fault rate, the plateau argument is wrong for thresholds, and the remaining contribution is only forward-only deployability.
Why not just use TM-NORM, which is already forward-only? It works only on models pre-trained with membrane-potential batch normalization. It also sets thresholds from input statistics, so nothing in it targets the objective, and it cannot compensate for faults that distort those statistics. The matched-pre-training comparison above settles this directly. If TM-NORM with its normalization layers equals our method under faults, the practical case narrows to legacy models.
Why not just run a stock evolution strategy with a big population? A large enough population might sample off the plateau by brute force, and the unmodified evosax baselines test exactly this. We expect stock methods to shrink their step size on plateaus and to waste the population on tied candidates. The population-size sweep will show whether our tie-breaking and drift reach a given accuracy with fewer evaluations, or whether scale alone erases the gap.
Proposed Experiments
We will use spiking image classifiers under standard input corruptions, with stuck-at faults injected at 0–20% of synapses. None of these setups come from the research file, which reports none for this task. We will add frozen spiking controllers on the Brax continuous-control tasks from the MoRobtrol suite introduced in Bridging Evolutionary Multiobjective Optimization and GPU Acceleration via Tensorization, where faults turn return into a near-flat landscape of the kind described by Gomez et al.. No reported result found on Lacuna for spiking policies with faults on those tasks.
Metrics are accuracy or return recovered relative to the fault-free network, evaluations needed to leave the plateau, wall-clock time, and measured memory efficiency.
Three checks are decisive. The first is whether the fraction of tied candidates in the population actually exceeds one half early in search; if it does not, the problem is not flat and the premise fails. The second is whether the surrogate-gradient baseline's accuracy falls with fault density while ours does not. The third is whether the population-minor layout keeps the cost of the sparse event path nearly flat as population size grows. A negative answer on the third leaves the search method intact but removes the case for large populations.
References
- Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks
- Practical Mechanism for Fault-Tolerant Spiking Neural Networks via Simple Input Control Based on Learnable Fragmentation
- evosax: JAX-Based Evolution Strategies
- Brian2CUDA: Flexible and Efficient Simulation of Spiking Neural Network Models on GPUs
- Column-Oriented Datalog on the GPU
- Generating Gradients in the Energy Landscape Using Rectified Linear Type Cost Functions for Efficiently Solving 0/1 Matrix Factorization in Simulated Annealing
- On the Effects of Smoothing Rugged Landscape by Different Toy Problems: A Case Study on UBQP
- Population control meets doob's martingale theorems: the noise-free multimodal case
- Speeding Up Hyper-Heuristics With Markov-Chain Operator Selection and the Only-Worsening Acceptance Operator
- Bridging Evolutionary Multiobjective Optimization and GPU Acceleration via Tensorization
- Solving Non-Markovian Control Tasks with Neuro-Evolution