Many physical networks carry signals that live on edges or faces rather than nodes: water flow in pipes, current in circuits, traffic on streets, ocean currents on a triangulated sea. To store such a signal you must pick an arbitrary arrow for every edge, and flipping that arrow flips the sign of the number. Neural networks that respect this convention are called orientation-equivariant. When an edge's arrow is reversed, their output on that edge flips sign and nothing else changes. Every such network in the literature is local. It passes messages only between edges that share a node or a triangle. That is a problem because the physics of these systems is often global. Kirchhoff's laws make the current in one wire depend on every conductance in the circuit, and the pressure that keeps a fluid incompressible depends on the entire velocity field.
This note proposes a global attention layer for edge and face signals that is orientation-equivariant by construction and runs in time linear in the number of simplices. The key observation is simple. Take the outer product of a key and a value that are both odd (sign-flipping) functions of the same edge's signal. The two sign flips cancel, so a sum of such products over all edges is unaffected by any choice of arrows. It can therefore serve as a global memory that any edge can read with an odd query. This removes the need for a "relative orientation" between distant edges. No canonical relative orientation exists for edges that share no triangle, and its absence is why current signed attention stops at one hop.
Core Claim
Let $x_j$ be the orientation-equivariant features of simplex $j$ (they flip sign when its arrow flips). Let $u_j$ be its orientation-invariant features, such as pipe diameter, conductance, or midpoint coordinates. Define the query, key and value as odd-linear maps of $x$, multiplied by gates computed from invariant features: $q_i = W_q x_i \odot g_q(u_i)$, and similarly $k_j$ and $v_j$. The global layer is
$$ y_i = \frac{q_i^\top S}{\sqrt{q_i^\top C\, q_i} + \epsilon}, \qquad S = \frac{1}{N}\sum_{j} k_j v_j^\top, \qquad C = \frac{1}{N}\sum_j k_j k_j^\top , $$
where $N$ is the number of simplices, $S$ is the global key–value memory and $C$ is a key second-moment matrix used for normalization. If arrow flips are written as a diagonal sign matrix $D$, then $S$ and $C$ are unchanged under $x \mapsto Dx$, because each term contains the product $d_j^2 = 1$. The denominator is even in $d_i$ and the numerator is odd, so $y \mapsto Dy$ exactly. The layer is a cubic, odd, nonlinear function of $x$. It therefore meets the requirement of Principled Simplicial Neural Networks for Trajectory Prediction, which showed that equivariant simplicial networks need odd nonlinearities because linear ones lose curl information. The memory $S$ is invariant, so it can pass through an arbitrary multilayer perceptron before being read.
The layer resolves the tension, listed as Gap 3 in the research file, between softmax attention and orientation equivariance. Softmax is positive and neither even nor odd. Simplicial Attention Networks (SAT) got around this only locally: it uses an even attention score times an explicit sign $o_{\sigma,\tau}$, which is defined only for simplices that share a face. Our layer drops softmax and handles normalization with an even quadratic form. Costs are $O(N d^2)$ for feature width $d$, the same profile as linear attention.
The invariant gates are what make the layer more than a fixed global average. Suppose they carry random Fourier features of simplex positions, or orientation-free structural encodings when no geometry exists. The diagonal of powers of the Hodge Laplacian is one such encoding, since $D L D$ has the same diagonal as $L$. Then the effective kernel between edges $i$ and $j$ depends on where both edges sit and on their local invariant properties. This is the ingredient a fixed Green's-function operator cannot supply.
Research Program
The architecture interleaves local and global blocks. The local block is a Hodge-aware simplicial convolution in the style of Hodge-Aware Convolutional Learning on Simplicial Complexes (SCCNN). SCCNN builds multi-hop polynomial filters from the separate "down" and "up" parts of the Hodge Laplacian and shows this avoids collapse into the harmonic subspace. We keep it for local gradient and curl structure. The global block is the layer above. It is applied per dimension (edges, faces) with several heads, and residual connections pass through odd nonlinearities such as $\tanh$.
Invariant/equivariant fusion follows Graph Neural Networks for Edge Signals: Orientation Equivariance and Invariance (EIGN), which processes the two signal types jointly but only locally. Its authors explicitly name global propagation as open, and our block is a direct answer. Our design also differs from Logic and the 2-Simplicial Transformer, which is global over triangles but ignores signs entirely. That paper also reports gradient instability from its degree-three attention. We therefore treat the cubic form of our layer as a known risk, and will monitor the conditioning of $C$ and gradient norms. The fallback is a spectral-norm clip on $W_k$.
The theory component is modest. We will prove exact equivariance and permutation equivariance. We will also show that with position gates the layer can represent any separable low-rank kernel. Finally, we will test empirically how the rank of $S$ (heads times width) limits how accurately it approximates the weighted Laplacian inverse. That inverse is the target the global block should learn on elliptic tasks.
Baselines
The naive reference points are predicting zero flow and a per-edge MLP on invariant features. Every model must clear these by a wide margin on the non-local tasks, or the tasks are not measuring what we think.
The strongest local equivariant competitors are SCCNN and EIGN. SCCNN is the closest method that deliberately widens the receptive field, using polynomial filters of the Hodge Laplacian. EIGN is the strongest reported method on edge flows with mixed directed and undirected edges; it reports RMSE reductions "up to 23.5%" over line-graph GNNs on four transportation networks. The file gives no absolute RMSE, so we will rerun EIGN's released configuration.
SaNN: Simple Yet Powerful Simplicial-aware Neural Networks (SaNN) precomputes multi-hop aggregations and reports 35.0% accuracy on Ocean Drifters versus 30.0% for the next best. SAT reports about 93.8% on the synthetic trajectory task with holes. Principled Simplicial Neural Networks for Trajectory Prediction (SCoNe) reports 0.92 on Berlin and 0.59 on reversed synthetic trajectories. For MeshFT-Net (Mesh Field Theory: Port–Hamiltonian Formulation of Mesh-Based Physics), there is no reported result on Lacuna for any edge-flow projection task. We include it as a structure-preserving local baseline on our mesh tasks.
A fixed global operator baseline feeds an SCCNN with extra channels. These channels are the input flow after applying the pseudo-inverse of the Hodge 1-Laplacian and after projecting onto its gradient, curl and harmonic parts. We compute them with sparse iterative solvers, in the spirit of the nearly-linear solvers of Hodge Decomposition and General Laplacian Solvers for Embedded Simplicial Complexes.
A non-equivariant global baseline is a standard linear-attention transformer on the line graph, trained with random arrow-flip augmentation.
All models get the same data splits and parameter budgets within 10%. Each gets the same hyperparameter search budget (about 30 trials per model per task) and the same seeds.
Three ablations isolate the key ingredient. The first replaces our normalization with softmax over $q_i \cdot k_j$, which breaks equivariance. The second removes the invariant gates. The third removes the global block entirely. Beating the baselines means lower error than both SCCNN and the pseudo-inverse-augmented SCCNN on the non-local tasks at matched compute, with 95% confidence intervals over five seeds. It also means no loss beyond noise on the local trajectory benchmarks.
Why not just…?
Why not just use a standard global transformer with sign-flip augmentation? Augmentation only makes equivariance approximate, and the evidence in the file is that unconstrained models learn storage conventions. SCoNe reports that an RNN drops to 0.01 accuracy on reversed trajectories while the equivariant model keeps 0.59. MeshFT-Net reports that unconstrained MeshGraphNets drift to an energy error of 25.9 versus $1.3\times10^{-4}$. Our experiments test this directly. We measure output deviation under random test-time arrow flips, and accuracy when evaluation graphs use orientation conventions never seen in training (for example, all arrows pointing "north").
Why not just raise the polynomial order of SCCNN or the hop count of SaNN? SaNN reports that going to 3 hops adds structural noise and that Laplacian powers cause out-of-memory errors on dense complexes. More fundamentally, a degree-$p$ filter still reaches only $p$ hops. Our size sweep, which holds the network fixed and grows mesh diameter, should show polynomial models degrading once the diameter exceeds their order while the global block does not. If SCCNN at high order matches us at equal wall-clock time, the global block is not justified, and we will report that.
Why not just precompute the Laplacian pseudo-inverse and feed it as a feature? This is the most serious threat. When the governing operator is fixed across samples, as in the unweighted Leray projection, a fixed Green's operator is exactly the right answer, and we expect this baseline to tie or win there. The proposal therefore stakes its claim on tasks where the global operator changes with invariant inputs. In a Kirchhoff network with per-sample conductances, the solution requires the inverse of $B_1 W B_1^\top$, where $B_1$ is the node–edge incidence matrix and $W$ is diagonal and varies per sample. With diodes, the relation is nonlinear. A fixed pseudo-inverse cannot track $W$. Our gated keys can, because they are functions of the conductances. The gap between these two regimes is the decisive result.
Proposed Experiments
The first benchmark is a controlled suite of Delaunay meshes with holes, following the synthetic setup of SCoNe and SAT. Mesh sizes are swept so the diameter grows about fourfold. Three edge-flow targets are defined. The first is the divergence-free projection of a random flow, a discrete analogue of the pressure-Poisson step that Efficient Vision Transformer-based Surrogate for Scalable Pressure Prediction in Incompressible Turbulent Flows found requires global attention on grids. The second is the current in resistor networks with per-sample conductances. The third is the same with diode edges. The metrics are relative RMSE and error as a function of the graph distance between source and measured edge. We also measure divergence violation, which is $\|B_1 y\|$, the net flow left at each node.
The second benchmark is the LTSpice circuit and transportation benchmarks of EIGN, where we report RMSE relative to EIGN. The third is a sanity check on local tasks: Ocean Drifters (reference: SaNN 35.0%), synthetic trajectories with holes (reference: SAT about 93.8%, SCoNe 0.59 on reversed), and Berlin (reference: SCoNe 0.92). A global block should not hurt on these.
There are three decisive checks. First, exact equivariance: the flip-deviation metric must be at machine precision for our model and nonzero for the augmented transformer. Second, on variable-conductance circuits the proposal must beat the pseudo-inverse-augmented SCCNN; a tie there falsifies the claim that input-dependent global routing matters. Third, error versus distance must stay flat for our model while rising for local models, with wall-clock and memory reported so the linear-time claim is checked against the $O(N\sqrt N)$ and $O(N^2)$ references set by CCNet: Criss-Cross Attention for Semantic Segmentation and dense attention.
References
- Principled Simplicial Neural Networks for Trajectory Prediction
- Simplicial Attention Networks
- Hodge-Aware Convolutional Learning on Simplicial Complexes
- Graph Neural Networks for Edge Signals: Orientation Equivariance and Invariance
- Logic and the 2-Simplicial Transformer
- SaNN: Simple Yet Powerful Simplicial-aware Neural Networks
- Mesh Field Theory: Port–Hamiltonian Formulation of Mesh-Based Physics
- Hodge Decomposition and General Laplacian Solvers for Embedded Simplicial Complexes
- Efficient Vision Transformer-based Surrogate for Scalable Pressure Prediction in Incompressible Turbulent Flows
- CCNet: Criss-Cross Attention for Semantic Segmentation