Event cameras and spiking neural networks (SNNs, models whose neurons communicate with discrete spikes over time rather than continuous activations) are meant to run on small edge devices. Those devices often learn together by exchanging model updates over wireless links that only work in one direction, for example because transmit power differs between devices. Recent work shows these models can be broken by moving the timing of a few spikes while leaving their number and location unchanged. Retiming fewer than 2% of spikes yields over 90% attack success on DVS-Gesture (Time Is All It Takes). The only defense shown to work is adversarial training, meaning training on attacked inputs, and it is costly. Nobody has asked whether that defense can be carried out across a network of devices that see different data and talk over one-way links.
We propose to find out, and we expect the straightforward combination to fail in a specific, measurable way. The standard way to train over one-way links with differing local data is gradient tracking. Each device keeps a running estimate of the network-average gradient and updates it with the change in its own gradient between rounds. Under adversarial training, that change has two sources: the weights moved, and the attacker's discrete retiming plan jumped to a different solution. The second source is not small. It enters every device's tracker and keeps circulating through the network. Our fix is to compute each device's gradient change on the same retiming plan at the old and new weights. We keep a per-example memory of plans that the attacker warm-starts from, so that the gradient change reflects only the weight change.
Core Claim
Gradient tracking over directed graphs can carry adversarial training of SNNs against spike-retiming attacks without a large loss of either clean or robust accuracy relative to centralized training. This holds only if the tracker's gradient differences are anchored to a fixed adversarial plan. If they are not, plan switches dominate the tracker's updates, and consensus error and robust accuracy both degrade as data heterogeneity grows.
The novelty lies in this diagnosis and in the anchored estimator for a discrete, per-example, non-concave inner attack over a directed graph. Directed gradient tracking itself is not new, and neither is variance reduction for decentralized minimax problems. We say plainly which pieces we reuse.
Where This Sits
Three bodies of work motivate the question.
- Centralized SNN defenses. Time Is All It Takes defines the threat model with three budgets: per-spike shift, total shift, and number of moved spikes. It also introduces projected-in-the-loop (PIL) optimization, which computes gradients through soft shift probabilities and then projects back onto feasible integer retimings. It shows that event denoising filters cannot remove the attack and that adversarial training costs clean accuracy. SNN-RAT and Breaking Gradient Temporal Collinearity harden SNNs against pixel-intensity attacks, through orthogonal weight regularization and through decorrelating gradients across time steps. All three are centralized, and the latter two do not target retiming.
- Decentralized adversarial training. Decentralized Adversarial Training Over Graphs does adversarial training over networks. It uses plain diffusion and consensus on undirected graphs with ANNs, so nothing corrects for drift under heterogeneous data.
- Directed optimization. Two lines supply our method.
- ExtraPush combines push-sum (a protocol that corrects one-way averaging by carrying an extra weight variable and dividing by it) with gradient differences. Spod-GT extends directed push-pull tracking (row-stochastic mixing for weights, column-stochastic mixing for the tracker) to nonconvex losses with sporadic participation.
- DM-HSGD evaluates gradients at old and new iterates on the same sample (a STORM-style estimator) inside gradient tracking for minimax problems. Our anchoring borrows that estimator.
Every one of these assumes Lipschitz gradients. DM-HSGD additionally assumes an undirected graph and a strongly concave inner maximization over a shared dual variable. Our inner problem is the opposite on each count: a per-example, integer-valued, highly non-concave retiming solved approximately by a few PIL steps.
One finding constrains the design. Decentralized consensus can find flatter, more robust minima than centralized training (Decentralized Adversarial Training Over Graphs). That leaves open whether tracking's drift correction helps or hurts robustness in this regime. We treat it as an empirical question, not an assumption.
Research Program
Setting. There are $n$ devices on a strongly connected directed graph. Each holds non-IID event data and trains a shared SNN, with weights $x_i$ at device $i$, using surrogate-gradient backpropagation through time. We build on push-pull tracking as in Spod-GT. Its updates use no division, which avoids the small-denominator instability of push-sum. We also run a push-sum variant to measure that instability directly.
Anchored estimator. Device $i$ stores a retiming plan $\delta_i(\xi)$ for each local example $\xi$. Each round it refreshes the plan by $m$ warm-started PIL steps at its current weights. It then forms
$$d_i^{k+1} = g_i(x_i^{k+1};\xi^{k+1},\delta^{k+1}) + (1-\beta)\big(d_i^{k} - g_i(x_i^{k};\xi^{k+1},\delta^{k+1})\big),$$
where $g_i(x;\xi,\delta)$ is the surrogate gradient of the loss on example $\xi$ retimed by plan $\delta$, and $\beta\in(0,1]$ is a momentum weight.
The tracker $v_i$ is then updated by column-stochastic mixing plus the innovation $d_i^{k+1}-d_i^{k}$. Because both gradient evaluations in the correction term share $(\xi^{k+1},\delta^{k+1})$, a plan switch shifts the level of the gradient estimate but does not enter the difference. Setting $\beta=1$ removes anchoring and recovers ordinary stochastic tracking, which serves as the key ablation.
Analysis. We will measure the quantity the diagnosis predicts: the tracker innovation norm, split into a weight-change part and a plan-change part. We will check whether the plan-change part grows with heterogeneity and correlates with consensus error.
On the theory side, we will prove a conditional result. Assume the surrogate gradient is $L_s$-Lipschitz in the weights for a fixed spike input; we will estimate $L_s$ empirically rather than claim it. Under that assumption, the anchored innovation is bounded by $L_s\|x^{k+1}-x^k\|$ plus a term proportional to $\beta$ times the plan-switch magnitude. This gives a push-pull convergence statement in the style of Spod-GT, with a bias floor controlled by $\beta$ and the refresh depth $m$. We will state openly that the assumption is the weak point, because spikes can flip under small weight changes.
Cost. Anchoring doubles backward passes per step but adds no communication, because plans never leave the device.
Baselines
Centralized reference (upper bound). PIL adversarial training of the same architecture on pooled data. This is the defense of Time Is All It Takes, which reports DVS-Gesture clean accuracy falling from about 95% to about 69–72% after adversarial training. That is the strongest reported point on the robustness/clean-accuracy trade-off. Matching its curve within a small margin, at equal total PIL steps and gradient evaluations, is our success criterion. We found no reported result on Lacuna for decentralized SNN training of any kind on DVS-Gesture, CIFAR10-DVS, or N-MNIST, so the decentralized baselines must all be run by us.
Naive reference. Each device trains its own adversarially trained model with no communication, alongside plain clean decentralized training. This shows what cooperation buys.
Strongest existing decentralized method. Push-pull tracking as in Spod-GT with local PIL adversarial examples and ordinary stored-gradient tracking. On clean CIFAR-10 with ResNet-18, that paper reports a 7% to 40% accuracy improvement over AB/Push-Pull variants. Beating it means higher robust accuracy at equal communication bytes and equal gradient evaluations, with lower consensus error, on at least two of three datasets. To equalize compute, the baseline receives the extra backward passes as additional PIL steps.
Other decentralized baselines. - Subgradient-push with local adversarial training, which uses no tracking. - Adversarial diffusion training from Decentralized Adversarial Training Over Graphs, run on a symmetrized version of each graph with its extra handshake round counted in communication. - Push-pull with per-device gradient normalization, following Decentralized Nonconvex Optimization under Heavy-Tailed Noise.
Ablations. Three ablations isolate the key ingredient:
- anchored versus unanchored differences, via $\beta$;
- warm-started versus freshly initialized plans;
- push-pull versus push-sum.
Every method gets the same hyperparameter search budget over step size and momentum.
Why not just…?
Why not just use subgradient-push, or plain averaging, and skip tracking? In clean convex settings tracking wins clearly. ExtraPush reaches $10^{-8}$ error in about 3,000 iterations, while hand-tuned subgradient-push stalls between $10^{-2}$ and $10^{-4}$ (ExtraPush). Those results say nothing about nonconvex adversarial SNN training, however, where the flatter minima found by plain consensus might matter more. This is a real threat. Our heterogeneity sweep settles it: if subgradient-push matches robust accuracy at high heterogeneity, tracking is unnecessary here and we will report that.
Why not just symmetrize the links and use undirected decentralized adversarial training? Symmetrizing requires acknowledgment rounds that real broadcast-only links make slow. Undirected diffusion training also has no drift correction (Decentralized Adversarial Training Over Graphs). We run it with the handshake cost counted, so the comparison is made on bytes and wall-clock rounds rather than argued.
Why not just clip or normalize gradients instead of anchoring? If plan switches act like heavy-tailed noise, normalization may tame them at no extra compute. This is the most serious competitor. If normalized push-pull matches anchored tracking at equal gradient evaluations, we will drop the anchored estimator and report the diagnosis alone. We will also test combining the two.
Why not just avoid adversarial training and use a cheap regularizer or a filter? Event filters fail against retiming without destroying clean accuracy (Time Is All It Takes). Temporal decorrelation targets directly encoded static inputs, not timing shifts on event streams (Breaking Gradient Temporal Collinearity). We nonetheless include push-pull with SNN-RAT-style orthogonal regularization as an inner-loop-free arm, since it sidesteps the instability entirely.
Proposed Experiments
Data and partitioning. We use DVS-Gesture, CIFAR10-DVS, and N-MNIST, with ConvNet and VGGSNN architectures as in the retiming paper. On DVS-Gesture we split data naturally by lighting condition. All datasets also get Dirichlet class splits at several concentration levels.
Graphs. Directed rings, directed exponential graphs, and sparse random directed graphs with 8 and 16 nodes, plus a Bernoulli-dropout variant following Spod-GT.
Metrics. - Clean accuracy. - Robust accuracy under PIL at each of the three budgets. These are evaluated with an ensemble of surrogate gradients, following the protocol of Adversarially Robust Spiking Neural Networks Through Conversion, so that gradient masking is not mistaken for robustness. - Consensus error. - The two-part tracker innovation norm. - Bytes communicated.
Decisive checks. Three outcomes test the core claim.
- The plan-change share of the innovation must rise with heterogeneity under unanchored tracking and fall under anchoring. If it does not, the diagnosis is wrong.
- Anchored push-pull must close most of the gap to centralized PIL training while unanchored push-pull does not, at equal compute.
- Normalized push-pull must fail to match anchoring. If it matches, the simpler method wins and we say so.
A secondary check compares push-sum against push-pull, to test whether dividing by small push-sum weights amplifies adversarial innovations on sparse graphs.
References
- Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networks
- SNN-RAT: Robustness-enhanced Spiking Neural Network through Regularized Adversarial Training
- Breaking Gradient Temporal Collinearity for Robust Spiking Neural Networks
- Decentralized Adversarial Training Over Graphs
- ExtraPush for Convex Smooth Decentralized Optimization over Directed Networks
- Sporadic Gradient Tracking over Directed Graphs: A Theoretical Perspective on Decentralized Federated Learning
- A Faster Decentralized Algorithm for Nonconvex Minimax Problems
- Decentralized Nonconvex Optimization under Heavy-Tailed Noise: Normalization and Optimal Convergence
- Adversarially Robust Spiking Neural Networks Through Conversion