Spiking neural networks, which pass sparse binary pulses between neurons instead of dense floating-point numbers, save energy because a pulse times a weight is just the weight: the chip only has to add, never multiply. That advantage stops at the output. To turn a spiking heatmap into sub-pixel coordinates, current practice either runs a non-differentiable peak search or falls back to a dense floating-point head. Predicting orientation, such as a ship's heading or a spacecraft's roll, is harder still, because a scalar angle wraps around (359° sits next to 0°) and naive regression on it blows up. We propose one readout that addresses both problems. Each spike in a local window adds its own fixed pixel coordinates, and their squares and cross-product, into six running sums. Dividing once gives the object's center and its spatial covariance. The orientation then follows from the covariance, with no angle parameter anywhere in the loss.
This readout is the soft-argmax idea (take the expected position under a heatmap) combined with the second-moment extension of 3D-Aware Object Localization using Gaussian Implicit Occupancy Function. The difference is that we normalize by spike counts rather than by a softmax, so apart from a single division per object it runs as accumulate-only arithmetic. We also use large-batch training as a stress test. Expectation-based losses are satisfied by many different heatmaps, including blurry and two-peaked ones. We ask whether suppressing gradient noise with large batches steers spiking networks toward those degenerate maps, and whether supervising the shape of the map prevents this better than generic large-batch optimizers do.
Core Claim
Let $c_t(p)$ be the spike count at pixel $p=(x,y)$ in timestep $t$ of the final spiking layer. Let $W$ be a local window around a candidate object, and let $N=\sum_{t,p\in W} c_t(p)$ be the total spike count in that window. The readout is
$$\mu=\frac{1}{N}\sum_{t}\sum_{p\in W} c_t(p)\,p,\qquad \Sigma=\frac{1}{N}\sum_{t}\sum_{p\in W} c_t(p)\,p\,p^{\top}-\mu\mu^{\top},\qquad \theta=\tfrac12\,\mathrm{atan2}\!\big(2\Sigma_{xy},\,\Sigma_{xx}-\Sigma_{yy}\big).$$
Here $\mu$ is the predicted center, $\Sigma$ is the $2\times2$ covariance, and $\theta$ is the orientation of the major axis. The coordinate constants $x, y, x^2, xy, y^2$ act as fixed synaptic weights, so every spike triggers six additions and no multiplications. Only the final normalization by $N$ and the $\mu\mu^\top$ correction are floating-point, once per object rather than once per pixel.
The loss is a 2D Wasserstein distance between the predicted Gaussian and a ground-truth Gaussian built from the oriented box. This loss is continuous in $\Sigma$, so it has no wrap-around cliff and no width-height swap. That second point matters: in oriented boxes, a small rotation can make the "width" and "height" labels exchange abruptly, which otherwise causes loss spikes for elongated objects. The loss is also symmetric under 180° rotation, as a line-of-orientation target should be. Training uses surrogate gradients through the counts.
We make three claims:
- This readout matches or beats a spiking backbone topped with a dense angle head on oriented localization, at a small fraction of that head's operation count.
- It sharply reduces the rotation errors seen on real event-camera spacecraft data.
- Its main failure mode, diffuse or bimodal count maps, gets worse under large-batch training. Supervising the map's shape repairs this better than layer-wise learning-rate scaling.
Research Program
What motivates the work. Four lines of evidence point to the gap:
- Soft-argmax gives sub-pixel precision from coarse maps. Integral Human Pose Regression raised MPII [email protected] from 17.2% to 29.3% by replacing argmax with an expectation. KNEEL used the same trick to avoid high-resolution output maps on radiographs. Both are floating-point networks with softmax normalization. We replace the softmax with a count normalization that spiking hardware can execute.
- Spiking detectors either avoid this machinery or stop short of orientation. Spiking CenterNet extracts centers by discrete peak search and predicts only axis-aligned boxes. Bio-Vision-Inspired SNNs for Object Detection with Event Cameras likewise reports axis-aligned detection. We reuse Spiking CenterNet's peak search only to place the windows, and differentiate through everything inside them.
- Where event-camera pipelines do face orientation, they fail. SPADES reports 78.98° rotation error for its keypoint-plus-PnP pipeline and 81.13° for direct regression on real data. These pipelines use frame-based networks on accumulated events, with no treatment of the angular boundary.
- The boundary-free methods we compare against all run on frame-based floating-point networks:
- Circular Smooth Label turns the angle into a classification over bins with a wrapped soft target.
- Dense Label Encoding compresses those bins with Gray codes.
- Deep Directional Statistics predicts unit vectors or von Mises parameters.
- E-DSNT predicts a Gaussian occupancy map and reads its moments.
What constrains the design. The known weaknesses of expectation readouts shape the method directly:
- An expectation averages separate peaks into a phantom point between them. We therefore read moments only inside peak-centered windows.
- A diffuse map defaults to the window center. We therefore add a Jensen-Shannon divergence term between the normalized count map and the target Gaussian, following E-DSNT, and gate on $N$: a window with too few spikes is reported as "no object" rather than as a centered guess.
- Truncation at image edges biases moments inward. We pad windows with virtual out-of-frame coordinates and measure the residual bias separately on objects near the border.
- AdaS shows that surrogate-gradient noise interacts badly with adaptive optimizers. We therefore report every result under both AdamW and momentum SGD.
What gets built. Everything else stays fixed across comparisons:
- Backbone: a ResNet-18 spiking encoder with parametric leaky integrate-and-fire neurons and threshold-dependent batch normalization. This normalization, from Going Deeper With Directly-Trained Larger SNNs, scales inputs relative to the firing threshold; that paper used it to reach 67.05% ImageNet top-1 in 6 timesteps. The backbone is the same across all comparisons.
- Readout: the six-accumulator moment readout.
- Operation counter: a tally of accumulate and multiply-accumulate operations per inference, split between backbone and head, using the same op-counting convention the spiking detection papers use. We do not import absolute joule figures that the file does not supply.
The large-batch study. On Large-Batch Training for Deep Learning found large-batch solutions 1 to 2 orders of magnitude sharper than small-batch ones and a CIFAR-100 test gap as high as 5%. It also found that augmentation did not close the gap but warm-starting with small batches did.
Our hypothesis is specific to expectation losses. Many count maps share the same moments, so the training loss has flat directions that include blurry and two-peaked maps. Small-batch noise may push the network toward compact peaks, which generalize because they are robust to event noise. Large batches may freeze wherever the moments first match.
A second effect is specific to spiking networks. Threshold-dependent batch normalization computes its statistics over the batch, so batch size also changes firing statistics. We separate this effect from gradient noise by also running with statistics frozen after warm-up.
Baselines
Naive reference points. Two spiking networks share the identical backbone:
- a Spiking CenterNet-style model that regresses a scalar angle with Smooth $\ell_1$;
- a CenterNet-style model with discrete argmax centers.
These show how much of the gain is the boundary-free target and how much is sub-pixel readout. Beating them is necessary but not interesting.
The strongest accuracy competitor. This is a Circular Smooth Label head attached to the same spiking backbone, run as a dense floating-point layer on the final membrane potentials. The design follows Arbitrary-Oriented Object Detection with Circular Smooth Label, which reports 96.10% mAP on HRSC2016 with a frame-based floating-point detector. That number is an upper reference from a full-precision network. We found no reported result on Lacuna for spiking oriented detection on HRSC2016. We also run the Gray-coded variant from Dense Label Encoding, which shrinks the head to 8 channels.
The strongest continuous competitor. This is a dense head predicting $(\cos 2\theta, \sin 2\theta)$ plus width and height. It follows the unit-vector representation of Deep Directional Statistics, with the angle doubled to respect 180° symmetry.
The ablations that isolate our key ingredient.
- Floating-point E-DSNT on the same spiking backbone, with softmax over accumulated membrane potentials in place of count normalization. This isolates whether count normalization costs accuracy.
- Our readout without the shape term. This isolates what the shape supervision contributes.
- A frame-based floating-point network with the E-DSNT head. This gives the full-precision ceiling.
Fairness rules. All models use the same data splits, augmentation, timestep count, and epochs. Each gets an equal hyperparameter search of the same number of trials over learning rate and loss weights. Results are averaged over three seeds.
What counts as beating a baseline. Our readout beats the Circular Smooth Label spiking head if it matches its mAP within seed noise while cutting head floating-point operations by at least an order of magnitude, or if it improves AP at the strict IoU 0.75 threshold at equal cost. On SPADES real data, beating the published 78.98° hybrid error by a wide margin is the bar. Matching the dense-head spiking baseline trained on the same data is the fair bar.
Why not just…?
Why not just keep a small dense angle head on top of the spiking backbone? This is the most serious threat. If the head's operations are a negligible fraction of the backbone's, the energy argument for an accumulate-only readout largely disappears. We will report the head's share of total operations at measured firing rates; Spiking CenterNet's 10.8–17.4% rates suggest the backbone is itself cheap, which makes the head share non-trivial. If that share turns out small, the contribution narrows to accuracy and simplicity: one map yields center, extent and angle, with no angle-specific head. The dense-head baselines above decide this directly.
Why not just regress $(\cos 2\theta, \sin 2\theta)$? This removes the wrap-around and handles 180° symmetry, so it is a real competitor on the angle alone. It still needs separate width and height outputs, which reintroduces the width-height swap for near-square objects. It also gives no sub-pixel center or uncertainty from the same map. If it matches our readout on HRSC2016 and SPADES, we will say so. In that case the remaining claim is the accumulate-only cost and the covariance-derived uncertainty, which we evaluate by correlating $\det\Sigma$ with error.
Why not just use LARS or small-batch warm-starting for large batches? Keskar et al. showed that warm-starting with small batches closes the gap in classification, so it will likely help here too; we include it. LARS is also included, following Revisiting LARS. Our prediction is narrower than "our method fixes large batches." The gap in expectation regression shows up as blurrier, more two-peaked test maps. Shape supervision targets that failure directly, and does so at no extra training cost, unlike warm-starting's slower schedule. If LARS alone closes the gap and restores compact maps, the hypothesis is wrong.
Why not just convert a trained floating-point network to a spiking one? Conversion needs long spike windows to approximate activations. That is the latency problem direct training with threshold-dependent batch normalization was designed to avoid. It also leaves the readout question untouched, because the converted head is still dense.
Proposed Experiments
Oriented detection on HRSC2016 and DOTA. Static images are presented for $T$ timesteps to the spiking backbone, as is standard for spiking networks on frame data. We report mAP, AP at IoU 0.75, mean absolute angle error, and head versus backbone operation counts. The decisive check is whether count-normalized moments lose accuracy relative to softmax moments on the same backbone. A loss of more than one mAP point would mean the accumulate-only restriction is costly.
Spacecraft on SPADES, synthetic and real. We use the readout in two ways:
- fitting the spacecraft's silhouette ellipse, following E-DSNT, which reported 0.93 IoU on AKM and cut orientation error from 10.2° to 0.1° in a floating-point network;
- sub-pixel keypoint localization feeding a PnP solver, as in Deep Keypoint-Based Camera Pose Estimation.
We report rotation error on real data against the 78.98° and 81.13° published figures and against our own dense-head spiking baseline. The decisive check is whether the synthetic-to-real rotation error drops for the moment readout specifically.
Sanity check on Gen1. On the axis-aligned Gen1 benchmark, the readout's diagonal covariance gives box extent. It should roughly match Spiking CenterNet's 0.229 mAP. A clear drop would indicate the readout harms standard detection.
Large-batch sweep. On HRSC2016 and SPADES synthetic, we sweep batch size from small to a large fraction of an epoch with linear learning-rate scaling. For each setting we record:
- the train-test gap;
- the Keskar sharpness metric;
- the entropy and a bimodality score of test count maps;
- firing rate, with threshold-dependent normalization statistics either live or frozen.
The prediction is that the gap grows with batch size and co-varies with map entropy more than with sharpness alone. The shape term should shrink both, more than LARS does. A null result, where the gap is flat in batch size, would itself be informative. It would suggest surrogate-gradient noise already supplies the exploration that large batches remove.
References
- 3D-Aware Object Localization using Gaussian Implicit Occupancy Function
- Integral Human Pose Regression
- KNEEL: Knee Anatomical Landmark Localization Using Hourglass Networks.
- Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detection
- Bio-Vision-Inspired Spiking Neural Networks for Object Detection with Event Cameras
- SPADES: A Realistic Spacecraft Pose Estimation Dataset using Event Sensing
- Arbitrary-Oriented Object Detection with Circular Smooth Label
- Dense Label Encoding for Boundary Discontinuity Free Rotation Detection
- Deep Directional Statistics: Pose Estimation with Uncertainty Quantification
- AdaS: Adaptive Gradient Descent for Spiking Transformers
- Going Deeper With Directly-Trained Larger Spiking Neural Networks
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
- Revisiting LARS for Large Batch Training Generalization of Neural Networks
- Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints