Standard Graph Neural Networks (GNNs) view the world through pairs. They pass messages along edges connecting two nodes. But real-world data often involves group relationships that cannot be neatly decomposed into independent pairs. For instance, a three-way chemical bond, a complex physical mesh, or a co-authorship triangle is fundamentally different from three separate pairwise connections. Furthermore, standard GNNs are limited in expressivity by the 1-Weisfeiler-Lehman (1-WL) isomorphism test, meaning they fail to distinguish certain non-isomorphic graph structures.
To overcome these limitations, a new paradigm has emerged: Topological Deep Learning (TDL). As highlighted by Papamarkou et al. in Topological Deep Learning is the New Frontier for Relational Learning, this approach "lifts" graphs into higher-dimensional topological spaces—such as simplicial complexes and cell complexes—enabling message passing not just between nodes, but across a rich hierarchy of multi-dimensional structures.
Lifting Graphs to Higher Dimensions
Before we can pass messages topologically, we must understand the spaces we are navigating. Instead of just vertices (0-cells) and edges (1-cells), we introduce higher-dimensional entities:
- Simplicial Complexes: Built from simplices. A 0-simplex is a node, a 1-simplex is an edge, a 2-simplex is a filled triangle, and a 3-simplex is a tetrahedron.
- Cell Complexes: A more flexible generalization where higher-dimensional "cells" can be bounded by arbitrary polygons or closed surfaces, making them ideal for modeling structures like chemical rings.
By treating these higher-order structures as distinct entities with their own features, we can model group-level interactions directly. For example, Hajij et al. in Cell Complex Neural Networks demonstrated that enabling information flow between cells of different dimensions allows a network to reason about multi-level organizations, such as moving from edges to faces.
The Mechanics of Topological Message Passing
In a standard GNN, a node only looks at its immediate neighbors. In a topological complex, a simplex or cell has a much richer neighborhood defined by four types of connectivity:
- Boundary Adjacency: Moving down a dimension. For example, a triangle communicates with the three edges that form its boundary.
- Co-boundary Adjacency: Moving up a dimension. An edge communicates with the triangles or higher-order cells it helps enclose.
- Lower Adjacency: Communicating with peers of the same dimension that share a common lower-dimensional face (e.g., two edges sharing a vertex).
- Upper Adjacency: Communicating with peers of the same dimension that share a common higher-dimensional co-face (e.g., two edges that are both part of the same triangle).
This multidimensional connectivity is formalized through algebraic topology tools like incidence matrices and the Hodge Laplacian. As explored by Isufi et al. in Convolutional Filtering in Simplicial Complexes, incidence matrices act as boundary operators that map signals between adjacent dimensions, allowing information to flow seamlessly across topological levels.
Architectures and Expressive Power
Several foundational frameworks implement these rules to build neural networks on top of topological spaces:
- Message Passing Simplicial Networks (MPSNs): Proposed by Bodnar et al. in Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks, MPSNs implement communication protocols across all four adjacencies. The authors proved that this framework is strictly more expressive than standard GNNs and can solve structural problems that even the powerful 3-WL test fails to resolve.
- CW Networks (CWNs): Extended by Bodnar et al. in Weisfeiler and Lehman Go Cellular: CW Networks, CWNs lift graphs to regular cell complexes, allowing hierarchical message passing that excels at capturing ring-like structures in molecules.
- Simplicial Attention Networks (SAT): Developed by Goh et al. in Simplicial Attention Networks, SAT introduces a signed attention mechanism over simplicial complexes, dynamically weighting higher-order interactions while maintaining orientation equivariance.
To make these models computationally viable, Gurugubelli proposed SaNN: Simple Yet Powerful Simplicial-aware Neural Networks, which pre-aggregates structural features recursively over multiple hops. This allows the model to capture wide topological contexts with constant run-time and memory requirements during training.
Why It Works: The Intuition of Joint Processing
Early attempts at processing higher-order data often filtered each dimension in isolation. For example, a model might run one convolution on nodes and a completely separate convolution on edges. However, Defferrard et al. noted in Simplicial Neural Networks that isolating dimensions prevents data from flowing between different levels of the complex.
The true power of modern topological message passing lies in joint processing. By using unified operators—such as the simplicial Dirac operator leveraged by Battiloro et al. in Generalized Simplicial Attention Neural Networks—information from nodes, edges, and triangles is integrated simultaneously. This respects the underlying physical and structural constraints of the data, allowing the network to capture complex geometric relationships like curl and divergence in physical systems, or multi-word phrases in short-text classification.
Key Foundational Work
Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks (2021) — contributed 3 concept elements to this direction.
Simplicial 2-Complex Convolutional Neural Networks (2020) — contributed 2 concept elements to this direction.
Cell Complex Neural Networks (2020) — contributed 2 concept elements to this direction.