Equivariance at the Boundary: Spiking Networks on Local Reference Frames

Spiking neural networks save energy because their neurons send binary pulses ("spikes"), so each connection performs a cheap addition only when a spike arrives. They avoid the multiply-accumulate operation that dominates the cost of ordinary networks. Recent spiking networks for 3D point clouds report a few millijoules per ModelNet40 classification. That efficiency rests on an assumption these papers never test: every object arrives in a canonical upright pose. The networks read raw $(x, y, z)$ coordinates, so rotating the input changes everything they compute. Conventional ANNs built this way collapse from about 89% to about 16% accuracy under arbitrary rotations (CRIN). The usual cure in the geometric-learning literature is an equivariant architecture, one whose internal features rotate predictably when the input rotates. The most expressive of these rely on dense, high-precision tensor products, which is exactly the arithmetic a spiking chip cannot do.

This note proposes a way out and a test of whether it is needed. A network can confine all rotation-dependent arithmetic to two thin boundary layers. At the input, each point's neighborhood is expressed in a small local coordinate frame built from the points themselves, which yields rotation-invariant numbers. At the output, predicted vectors are rebuilt as combinations of those same frame vectors. Everything between these layers operates on invariant scalars and can be fully spiking. The output layer can also be made addition-only: each output spike adds a precomputed frame vector to the result. The question is whether this local-frame design earns its place against the obvious cheaper option. That option is to rotate the whole cloud into a canonical pose once and feed it to an existing spiking network.

Core Claim

Our claim is that local-frame scalarization makes the overhead of equivariance in a spiking network small and localized. It also keeps the accuracy advantages of local rather than global frames. We expect those advantages to show up on partial and cluttered scans and on tasks with vector or rotation outputs. Global canonicalization cannot give such tasks correct equivariant outputs without extra machinery.

The structural observation behind the proposal is as follows. Let $e^{(1)}_{ij}, e^{(2)}_{ij}, e^{(3)}_{ij}$ be an orthonormal frame on the edge from point $i$ to neighbor $j$. Let $s_{ijk}(t)\in\{0,1\}$ be the spike emitted at time step $t$ by output channel $k$ on that edge, and let $w_{k,m}$ be learned scalar weights. A predicted vector at point $i$ can then be written as

$$ v_i \;=\; \sum_{t=1}^{T}\sum_{j\in\mathcal{N}(i)}\sum_{k}\, s_{ijk}(t)\sum_{m=1}^{3} w_{k,m}\, e^{(m)}_{ij}, $$

where $T$ is the number of time steps and $\mathcal{N}(i)$ is the neighborhood of $i$. The inner sum is a fixed vector per edge and channel, computed once when the frame is built. Every spike therefore triggers only an addition of that vector.

Because the frames rotate with the input, $v_i$ is exactly rotation-equivariant regardless of how coarsely the spikes quantize it. This holds as long as the frames themselves are well defined, and handling the cases where they are not is part of the program. The only multiplications left in the network are the frame construction and the projection of relative coordinates onto frames. Spiking-network energy accounting already assigns those operations to the first, non-spiking encoding layer.

Relation to Prior Work

Three spiking point-cloud networks motivate the gap. Spiking PointNet runs leaky integrate-and-fire neurons in a PointNet backbone, trained at a single time step. Spiking Point Transformer (SPT) builds spike-based local attention with a queue of furthest-point-sampled subsets across time steps. 3DSMT couples spiking offset attention with spiking Mamba blocks. All three are evaluated only on aligned data and encode coordinates in a frame-dependent way. We keep their training recipes and replace their coordinate input with frame-projected scalars.

The method comes from ClofNet. ClofNet builds per-edge orthonormal frames with cross products, projects vectors onto them, runs unconstrained message passing on the resulting invariants, and projects back. It keeps universal approximation and runs about 4× faster than SE(3)-Transformers. We make its interior spiking and its projection-back step addition-only. We inherit its known weakness: frames degenerate when points are collinear.

CRIN plays two roles. Its centrifugal reference frame is cheap (1.74 ms versus about 8 ms for PCA-based frames) and reaches 91.8% on rotated ModelNet40. That makes it the strongest competitor, because it could simply sit in front of an existing spiking network. But its frames hinge on the global center of the cloud, which the file flags as fragile under occlusion and truncation. That fragility is what our experiments probe.

Other equivariant designs constrain the choice rather than supplying it. Steerable networks built on Clebsch-Gordan products (Asymptotically Fast Clebsch-Gordan Tensor Products) need continuous products of high-order features and do not fit accumulate-only hardware. EGNN avoids those products but cannot represent directions off the line between two particles. Discretized designs like E2PN multiply memory by the number of rotation channels. Local frames are the one route that yields invariant scalars without these costs.

The only spiking work touching symmetry, Neuromorphic Visual Scene Understanding with Resonator Networks, handles 2D transformations with phasor codes on Loihi (171× lower energy-delay product than a CPU). It does not extend to 3D point sets. Geometry-Aware Spiking GNN runs spiking dynamics on curved manifolds for citation graphs and enforces no spatial symmetry. Neither overlaps with our design.

Research Program

The model, which we call Spiking Local-Frame Network, follows a hierarchical point-cloud layout. It uses k-nearest-neighbor grouping, with sampling chosen to be rotation-stable, since the file notes that SPT's furthest point sampling is unstable under rotation.

For each point we build a local frame from three quantities: the offset to the local neighborhood centroid, the offset to a neighbor, and their cross product. This follows ClofNet's edge frames. Projecting relative coordinates, and optionally local covariance statistics, onto the frame yields invariant scalars. These scalars enter a direct-encoding layer, which is the only multiply-accumulate layer before the output. Subsequent layers are integrate-and-fire or leaky integrate-and-fire message passing and aggregation on spikes. We reuse SPT's hybrid-dynamics neurons and the single-step training with membrane-potential noise from Spiking PointNet, so that the backbone, not the recipe, is what changes.

The heads depend on the task. Classification and part-segmentation heads read spike counts, so their outputs are invariant by construction. For equivariant outputs we use the spike-accumulated vectorization above. Examples are per-point displacement vectors and a global rotation assembled from two predicted equivariant vectors by Gram-Schmidt orthogonalization.

Frame degeneracy is handled explicitly. When the centroid offset and neighbor offset are nearly collinear, we fall back to the next neighbor in distance order. We will report how often this happens and whether it breaks exact equivariance numerically.

Energy accounting follows the synaptic-operation convention used across this literature (0.9 pJ per addition, 4.6 pJ per multiply-accumulate at 45 nm). It separately itemizes three costs: frame construction, projection, and output vector accumulation. We treat this as a theoretical estimate, per the caveat raised in 3DSMT. Deployment on real neuromorphic hardware is out of scope for the first phase.

Baselines

The naive reference point is the published aligned-data spiking networks, SPT and Spiking PointNet (3DSMT if code is available), trained on upright data and tested under the protocols below. On aligned ModelNet40 these report 91.43% at 3.0 mJ (SPT) and 95.2% at 4.3 mJ (3DSMT). No reported result was found on Lacuna for any spiking point-cloud network under arbitrary rotation. We expect near-collapse, as for PointNet, but we will measure it rather than assume it.

The strongest competitor is global canonicalization followed by an unmodified spiking network. Concretely, each cloud is rotated into CRIN's centrifugal reference frame and passed to SPT, with the same training recipe and neuron model as ours. It is the strongest because it adds only one cheap frame per cloud, versus our one frame per edge. It also inherits CRIN's accuracy, which reports 91.8% on ModelNet40 trained upright and tested rotated, and 84.7% on ScanObjectNN. The file does not specify the rotation protocol for the ScanObjectNN figure, so we will rerun CRIN under our protocol rather than quote it.

Beating this baseline means two things. First, matching its accuracy on full, rotated clouds within one point. Second, exceeding it on truncated and occluded clouds and on rotation estimation, with total theoretical energy, including frame costs, no more than 1.5× higher.

We add an ANN reference: our identical local-frame architecture with continuous activations. It sets the accuracy ceiling and the energy that spiking must undercut.

We also include an ANN-to-SNN converted version of that ANN at several time-step counts. It tests whether direct spike training is needed at all.

Two ablations isolate the key ingredients. The first swaps local frames for the global CRF inside our backbone, which isolates locality. The second replaces the spike-accumulated vectorization with a continuous output layer, which isolates the cost and precision of addition-only equivariant outputs.

All models get the same data splits, point counts, epochs, and hyperparameter search budget (a fixed number of trials each), and energy is counted with one script.

Why not just…?

Why not just train an existing spiking network with heavy rotation augmentation? Augmentation offers no guarantee on unseen poses. Poulenard et al. and CRIN both argue it is costly and covers SO(3) poorly. It also multiplies training volume for networks whose surrogate-gradient training is already fragile. It is still a fair competitor, so we include SPT trained with random SO(3) rotations at an equal-compute budget and report both mean accuracy and the spread of accuracy across poses. If it matches us on accuracy, the case for our model on invariant classification rests on consistency across poses and on training cost, and we will say so.

Why not just canonicalize the whole cloud once and keep the spiking network as is? This is the real threat, and on clean, complete ModelNet40 clouds we expect it to roughly tie with us. The case against it is specific. A global frame depends on the cloud's center and extent, so cropping or occlusion moves the frame and changes every downstream feature. The file flags this dependence as CRIN's weakness, but it is an argument, not a measured result. Our truncation sweep and the ScanObjectNN hardest split with rotations will settle it. If the global frame holds up under truncation, the proposal narrows to equivariant-output tasks only.

Why not just convert a trained equivariant ANN to spikes? Rate-coded conversion needs many time steps to represent continuous values. That multiplies latency and spike counts, eroding the 6–48× savings that directly trained networks report. It also quantizes coordinate updates in EGNN-style models, which breaks geometric consistency. Our design sidesteps the second problem because coordinates never pass through spikes, only invariants and frame coefficients do. The conversion baseline across time-step counts will show directly whether the first problem is real at the accuracy we need.

Why not just use rotation-invariant random features? Rotation-Invariant Random Features match deep equivariant networks on QM7 and ModelNet40 without a deep network at all. However, they become ill-conditioned on larger data, cost quadratic time in point count, and have no hierarchy for cluttered scenes or dense outputs. We include them on rotated ModelNet40 as a cheap reference, with energy counted as multiplies. If they match us there, our claim is limited to ScanObjectNN and segmentation.

Proposed Experiments and Decisive Checks

Classification runs on ModelNet40 and ScanObjectNN under three protocols: aligned train and test, upright train with random-rotation test, and random-rotation train and test. Part segmentation runs on ShapeNetPart under the same protocols and reports mean intersection-over-union.

The first decisive check is a truncation sweep. We crop 0–50% of each rotated test cloud by a random plane and plot accuracy against the crop fraction for our model, CRF+SPT, and the global-frame ablation. Our claim predicts the gap widens with cropping. If the curves coincide, locality buys nothing for invariant tasks.

The second decisive check tests equivariant outputs. We estimate the rotation between two copies of a ModelNet40 cloud, where CRIN reports 98.9% accuracy on its unsupervised rotation-estimation task. We measure angular error as a function of time steps $T$ and output channel count, to find whether spike-accumulated vectors are precise enough. The file identifies dense regression as a known weakness of spiking networks, so a failure here is plausible and informative.

Throughout, we report the per-stage energy breakdown. The central quantitative question is what fraction of total energy goes to frame construction and projection. If that fraction exceeds the savings from spiking, which is visible directly against the ANN reference, the boundary-equivariance idea fails on its own terms.

Finally, as a smaller transfer test on molecular graphs, we run QM9 property prediction with ClofNet-style edge frames. Here the risk is the collinearity fallback. Linear fragments are common in molecules, so the fallback rate matters.