Axial Global Kernels as Compressible Synapses for Spiking Vision

Spiking neural networks (SNNs) are networks whose neurons communicate with binary events over time. On neuromorphic chips, the main limit on their size is not arithmetic. It is the memory needed to store connections: for every synapse, the chip must know which neuron to deliver a spike to and with what weight. Chips stay small and low-power by exploiting a trick that only works for ordinary local convolutions. Because the same small kernel is reused at every position, the connection table collapses into a compact rule. Spiking vision models, however, increasingly need global spatial context, meaning a neuron can see distant parts of the image. The usual ways to get it are attention, mixer layers, or very large 2D kernels. These either break that reuse pattern or multiply per-spike traffic.

We propose a spatial mixing layer for SNNs designed around both constraints at once. It applies two one-dimensional kernels, one along rows and one along columns, each about twice as long as the feature map. Two such layers give every neuron a receptive field covering the whole image. The weight between two neurons still depends only on their offset along one axis, so the connectivity remains a sliding rule that a chip can store in $O(C(H+W))$ weights rather than in explicit tables. Row and column broadcasts also line up with the X-then-Y packet routing that 2D-mesh neuromorphic chips already use. The claim is that this layer recovers the dense-task accuracy that global context buys, at close to the synaptic memory of a 3×3 convolution. The claim fails if a simpler, position-blind global operator does just as well.

Core Claim and What Is New

Two facts from the literature motivate the work. First, Unveiling the Spatial-temporal Effective Receptive Fields of Spiking Neural Networks introduces a gradient-based diagnostic of which input pixels and timesteps influence an output. With it, the authors show that spiking transformers built on early convolutional mixers have a strong locality bias. Their fixes, a pixel-wise MLP mixer and a hybrid block called SRB, widen the receptive field. The paper does not count what those layers cost in synaptic storage or on-chip routing. Second, Darwin3 reaches 2.35 million neurons partly through its "Linker", which exploits weight sharing in convolutions to cut connectivity memory by 6.8× to 200.8×. The paper states that this compression is built for local sliding kernels. So the method that widens receptive fields and the method that saves synaptic memory currently work against each other.

The proposal reuses its mechanism from ParCNetV2. That non-spiking CNN uses 1D depthwise kernels roughly twice the input size, for example 111×1 on a 56×56 map. The kernel spans the whole axis from any position, and zero padding at the borders breaks shift invariance in a way that implicitly encodes absolute position.

Our observation is that, with the padding fixed, this operator is still a convolution. The synaptic weight from a source at column $x$ to a target at column $x'$ in the same row is $w_c[x'-x]$, where $w_c$ is a learned vector of length $2W-1$ for channel $c$. The routing rule needed is just "same row, any column." Every global spatial operator we found in the file breaks this offset-only property in at least one way: it uses weights that depend on content, weights that depend on position, or all-to-all fan-out.

Nobody has built this operator as a spiking layer, measured its synaptic table size under a neuromorphic storage model, or tested whether its position-aware geometry matters relative to cheaper global summaries. Those three steps are the contribution.

Research Program

Spiking axial mixer. We will build an axial spiking mixer. A row kernel and a column kernel act on spike maps of shape time × channels × height × width. Leaky integrate-and-fire neurons follow each kernel, and a 1×1 channel mixer completes the block. We will implement it in the QKFormer codebase (zhouchenlin2096/QKFormer) and the Spikingformer codebase (TheBrainLab/Spikingformer). In each, the axial mixer replaces the early convolutional mixers that the ST-ERF paper identifies as the locality bottleneck. Training uses surrogate-gradient backpropagation through time with the recipes those codebases ship.

Synaptic-memory accounting model. We will write an analysis tool that takes any trained SNN and reports three storage figures for each layer. The first is explicit point-to-point table size: a source index, target index, and weight per synapse, the regime Minitaur and early NEST faced. The second is a Linker-style shared-kernel size, which assumes weights shared across positions are stored once. The third is a rule-plus-weights size, in the spirit of Minitaur's ranged rules and procedural connectivity in BrainPy.

The tool also reports synaptic operations per input spike. It then simulates packet traffic on a 24×24 mesh with dimension-order routing, mirroring Darwin3's core layout. It uses a fixed tiling that maps image rows onto mesh rows. Under this tiling, a row broadcast becomes a single multicast along one mesh dimension.

We cannot run on Darwin3 silicon. The hardware claims are therefore cost-model claims, and we will report them as such. We will validate the shared-kernel accounting against Darwin3's published 6.8×–200.8× savings range on convolutional SNNs before trusting it on new layers.

Neuron dynamics and receptive field. We will apply the ST-ERF diagnostic as a function of membrane decay constant. This addresses that paper's own open question about how neuron dynamics shape receptive field growth. Fast decay could truncate the spatial spread across the two axial hops. If it does, that is a design constraint worth knowing.

Baselines

The naive reference is the unmodified Meta-SDT/Spikingformer-style backbone with its local convolutional mixers. It sets the floor on accuracy and the floor on synaptic memory, and every other method is measured relative to it. As the stacked-convolution variant, we will also give it extra 3×3 layers until its parameter count matches the axial model.

The strongest receptive-field competitor is SRB/MLPixer from the ST-ERF paper. It is the published spiking intervention aimed at exactly this locality bias, and it reports 48.9% AP50 on COCO 2017 and 43.7% mIoU on ADE20K. We will retrain it under our exact recipe, timesteps, and epoch budget. We will not copy those figures as targets, because recipes differ. Beating it means matching or exceeding its retrained mIoU and AP50 at substantially lower rule-plus-weights storage. A tie in accuracy with a several-fold storage reduction counts as a win for the memory claim.

The strongest global-context competitor is Q-K attention from QKFormer. It gets linear-cost global context by summing spike matrices into binary importance vectors, without Softmax or a value matrix. Its connectivity is also fairly regular, which makes it the most direct threat to our memory argument. The paper reports 85.65% ImageNet-1K top-1 at 64.96M parameters and 113.64 mJ, and 84.0% on CIFAR10-DVS with 1.50M parameters. We found no reported synaptic-table memory figure on Lacuna for QKFormer or any spiking transformer, so our accounting tool will produce that number for all methods. We will swap Q-K token attention in place of the axial mixer at identical depth, width, timesteps, and tuning budget. Each variant gets the same small learning-rate and weight-decay sweep.

For ImageNet we will compare at the Spikingformer-8-512 scale, which reports 74.79% at 224×224, T=4, and 29.68M parameters. We cannot afford QKFormer's largest configuration.

Two ablations isolate the key ingredient. The first, local axial, keeps the same layer but shortens the kernels to length 3. This removes global reach while keeping the structure and checks whether reach drives the gains. The second, circular axial, keeps global kernels but uses circular padding. This removes the implicit position encoding while keeping reach and checks whether position-aware geometry matters beyond what a position-blind global operator provides.

Why not just…?

Why not just stack more 3×3 convolutions? The ST-ERF diagnostic shows that stacked local mixers in spiking transformers leave early and middle neurons blind to distant context. Depth also costs training memory, which grows with depth times timesteps: Towards Zero Memory Footprint SNN Training reports ResNet-101 SNN training at around 29 GB. The parameter-matched stacked baseline tests this directly on ADE20K and in receptive-field maps.

Why not just use Q-K attention, which is already global and linear? This is the real threat. Its summation weights are uniform, so it may also compress into a rule. If it matches the axial mixer on accuracy at similar storage, our position-geometry argument is wrong. We expect it to lag specifically on dense and relational tasks. The reason is that it collapses tokens into a global importance vector, discarding where along an axis evidence came from; BCN's results (Broadcasting Convolutional Network) show such position cues matter for relational reasoning. The head-to-head on ADE20K and on a spiking Sort-of-CLEVR task settles this. The circular-padding ablation separates "global" from "position-aware".

Why not just use large 2D kernels and let a Linker compress them? A Linker-style scheme would compress a large 2D kernel. However, a global 2D kernel stores about $4HW$ weights per channel rather than about $2(H+W)$, and each spike fans out to the whole map instead of one row and one column. WBMM shows even dense GPUs suffer 71–78% slowdowns from the scattered memory reads of large 2D kernels. We will include a 31×31 spiking depthwise variant and report both its storage and its mesh traffic. If its traffic penalty turns out small in simulation, we will say so.

Proposed Experiments

Training benchmarks. All variants train on CIFAR10-DVS, ImageNet-1K at Spikingformer-8-512 scale, and ADE20K and COCO 2017 using the ST-ERF downstream setup.

Reported metrics. For each run we report accuracy, mIoU, or AP. We also report the three storage figures from the accounting tool, synaptic operations per spike, and simulated mesh packet-hops and peak link load.

First decisive check: storage. The axial mixer must land within about 2× of the local baseline's rule-plus-weights storage. Its explicit-table storage should exceed SRB/MLPixer by far less than its accuracy deficit, if it has one. The 2× threshold is our own target, not a published figure.

Second decisive check: dense-task accuracy. On ADE20K, the axial mixer must beat both the circular-padding ablation and the Q-K swap by a margin larger than seed variance over three seeds. If it does not, the position-aware claim fails and the proposal reduces to "any global operator works."

Third decisive check: neuron dynamics. We will plot ST-ERF coverage against membrane decay constant. If fast decay collapses coverage after the second axial hop, the layer needs explicit constraints on neuron parameters before it is deployable.