When we model relational data using standard Graph Neural Networks (GNNs), our focus is almost always on the nodes. But many real-world phenomena—such as water flowing through a network of pipes, traffic moving across city streets, or ocean currents circulating around islands—live inherently on the connections between nodes, or even on higher-order structures like triangles and polyhedra.
To represent these directional signals mathematically, we encounter a fundamental hurdle: we must assign an arbitrary reference direction to each connection. If water flows from point A to point B at 5 liters per second, we might write this as $+5$ if we orient the edge as $A \to B$. However, if we had arbitrarily chosen the reference orientation to be $B \to A$, that exact same physical flow would be written as $-5$.
As Michael T Schaub et al. point out in Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyond, these reference directions are purely human conventions. The underlying physical reality does not change when we flip them. If a neural network is not explicitly designed to handle this ambiguity, flipping an edge's reference orientation can lead to completely inconsistent or physically meaningless predictions. To solve this, modern architectures enforce orientation equivariance: a guarantee that if we flip the arbitrary reference direction of a spatial element, the network's internal representations and outputs will transform predictably (by symmetrically changing sign) rather than breaking down.
Why Standard Networks Fail
Standard GNNs are designed for node features or undirected edge features, meaning they lack the mathematical machinery to understand skew-symmetry—the property where a signal's value is intrinsically tied to its direction. As explained by the authors of HodgeNet: Graph Neural Networks for Edge Data, standard neural network components fail to respect this property, causing models to learn patterns tied to data storage conventions rather than actual physical relationships.
To build a truly orientation-equivariant network, two structural ingredients are required:
- Topological Linear Operators: Instead of using standard graph adjacency matrices, these networks utilize operators rooted in algebraic topology, such as the Hodge Laplacian or the Topological Dirac operator. In Simplicial Convolutional Neural Networks, Maosheng Yang et al. demonstrate that aligning mathematical operations with the Hodge structure ensures that linear filtering steps naturally scale when reference orientations are flipped.
- Odd Activation Functions: Consider a standard ReLU activation function, which maps all negative numbers to zero ($\max(0, x)$). If an edge orientation flips, a flow of $+5$ becomes $-5$. A ReLU would pass the $+5$ but completely erase the $-5$, destroying the physical signal and violating equivariance. To prevent this, models must use odd activation functions where $\sigma(-x) = -\sigma(x)$, such as the hyperbolic tangent ($\tanh$).
In Principled Simplicial Neural Networks for Trajectory Prediction, T Mitchell Roddenberry et al. formalize this as a principle of "admissibility." They prove that combining topological operators with odd nonlinearities prevents the network from learning artifacts based on specific edge orientations, dramatically improving generalization to unseen or reversed paths. This same mathematical requirement appears in coupled dynamical systems; Timotéo Carletti et al. show in Global Topological Dirac Synchronization that odd coupling functions are mandatory to keep the collective behavior of multi-dimensional oscillators independent of arbitrary directional labeling.
Scaling to Higher Dimensions: Simplicial Complexes
The concept of orientation is not limited to 1-dimensional edges. It extends naturally to 2-dimensional triangles (where orientation means clockwise vs. counter-clockwise), 3-dimensional tetrahedra, and higher-order structures known collectively as simplices.
Node (0-simplex) Edge (1-simplex) Triangle (2-simplex)
● ───> ● ● ───> ● ● ───> ●
\ / \ / ↺
\ / \ /
▼ ▼
● ●
In Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks, Cristian Bodnar et al. introduce Message Passing Simplicial Networks (MPSNs). By designing aggregation functions that explicitly account for the orientation signs of these higher-order structures, MPSNs can process vector fields and track global topological features—such as holes in a manifold—without sensitivity to how the simplices are indexed.
This orientation-aware processing enables powerful signal decomposition. The authors of Simplicial Convolutional Filters demonstrate that by leveraging the Hodge Laplacian, convolutional filters can independently isolate the gradient (divergence-like), curl (rotational), and harmonic (global) components of a flow. Because these filters maintain strict orientation equivariance, changing the reference direction of any simplex results in a consistent sign change in the output, preserving the physical meaning of the data across different coordinate representations (the authors of Convolutional Filtering in Simplicial Complexes).
Alternative Paradigms and Physical Invariance
While message passing is the dominant paradigm in graph representation learning, orientation equivariance can also be achieved through alternative geometric formulations. For instance, Kelly Maggs et al. introduce Simplicial Representation Learning with Neural $k$-Forms, which learns representations by integrating differential forms over geometric embeddings. Because integration is inherently signed, this framework naturally captures circulation and flow-based patterns that orientation-agnostic models completely miss.
Real-world applications, however, rarely consist of purely directional data. Models often need to process a mixture of directional flows (which must flip signs) and static physical properties like pipe diameters or distances (which must remain constant). To address this, Dominik Fuchsgruber et al. developed a framework in Graph Neural Networks for Edge Signals: Orientation Equivariance and Invariance that successfully disentangles and processes joint orientation-equivariant and orientation-invariant signals simultaneously.
Ultimately, hard-wiring orientation equivariance into architectures fundamentally changes how networks simulate physical reality. In Mesh Field Theory: Port–Hamiltonian Formulation of Mesh-Based Physics, the authors show that enforcing orientation covariance ensures that physical simulations naturally satisfy the action-reaction principle and conserve momentum without needing explicit global penalties. By respecting the underlying geometry of the space, architectures like SaNN (the authors of SaNN: Simple Yet Powerful Simplicial-aware Neural Networks) can remain computationally lightweight while retaining the exact mathematical expressiveness required to model complex trajectories, fluids, and vector fields accurately.
Key Foundational Work
Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks (2021) — contributed 5 concept elements to this direction.
Principled Simplicial Neural Networks for Trajectory Prediction (2021) — contributed 4 concept elements to this direction.
Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyond (2021) — contributed 3 concept elements to this direction.