Abstract
Traditional Graph Neural Networks (GNNs) are primarily limited to modeling pairwise relationships between nodes. This paper explores Simplicial Neural Networks (SNNs), a framework that utilizes simplicial complexes to capture higher-order interactions, such as multi-author collaborations. Using a massive paper-author bipartite graph derived from Semantic Scholar—comprising over 39 million papers and 12 million authors—we demonstrate how SNNs leverage the Simplicial Laplacian to perform convolutions on higher-dimensional data structures. Our results show that SNNs effectively impute missing citation data, significantly outperforming standard baselines and exhibiting strong filter transferability across different collaboration networks.
Introduction: The Limitation of Pairwise Modeling
While Graph Neural Networks (GNNs) have revolutionized relational data processing, they fundamentally rely on graphs that model interactions between pairs of nodes. In many real-world scenarios, interactions involve groups of three or more entities. For instance, a research paper co-authored by three people is more than just three separate pairs of collaborators; it represents a single cohesive unit of work. While hypergraphs can represent these sets, they often lack a rigorous topological framework to understand the underlying "shape" of the data. Simplicial Neural Networks (SNNs) address this by operating on simplicial complexes—mathematical structures that naturally encode multi-way interactions like triangles and tetrahedra.
Methods: Convolutions on Simplicial Complexes
The core of the SNN framework is the generalization of graph convolutions to higher dimensions using the spectral properties of the simplicial complex.
- Simplicial Complexes: Data is represented using $p$-simplices: vertices (0-simplices), edges (1-simplices), and triangles (2-simplices).
- Simplicial Laplacians ($L_i$): Similar to the graph Laplacian, the higher-order Simplicial Laplacian captures connectivity between different dimensions, allowing information to flow between edges and the triangles they bound.
- Spectral Convolution: Convolutions are defined in the frequency domain using eigenvectors of the Simplicial Laplacian. By using low-degree polynomials, the operation remains local and computationally efficient.
Dataset and Task
The model was evaluated on Coauthorship Complexes (CC1 and CC2) derived from the Semantic Scholar Open Research Corpus. This dataset contains 39,219,709 papers and 12,862,455 authors, forming 139,268,795 edges. In this representation, authors are vertices, and a paper with $k$ authors is represented as a $(k-1)$-simplex. The primary task is missing data imputation: predicting citation counts for specific collaborations based on partial information.
Key Results
SNNs were tested on their ability to impute missing citation data at rates ranging from 10% to 50%.
- Performance and Accuracy: SNNs significantly outperformed simple statistical baselines like the "Global Median" or "Neighbors Mean." As shown in the performance metrics, the model maintains high accuracy across different dimensions even as the percentage of missing data increases.
- Error Distribution: The absolute error distribution reveals that the vast majority of SNN predictions are highly accurate, with errors concentrated near zero.
- Transferability: One of the most striking findings was that filters learned on one complex (CC2) performed nearly as well on another (CC1), suggesting that SNNs capture universal patterns in how information flows through collaboration structures.
Discussion and Future Directions
While SNNs provide a principled topological approach to higher-order data, several challenges remain. Future work should focus on benchmarking against state-of-the-art hypergraph methods and developing layers that simultaneously process multiple dimensions (vertices, edges, and triangles). Additionally, scaling these complexes for even larger datasets through efficient pooling or coarsening operations will be critical for broader application in fields like fluid dynamics or global climate modeling.
Summary updated