Paper-author bipartite graph from Semantic Scholar

Stefania Ebli, Michaël Defferrard, Gard SpreemannarXiv 2020 · DOI · arXiv

Abstract

Traditional Graph Neural Networks (GNNs) are primarily limited to modeling pairwise relationships between nodes. This paper explores Simplicial Neural Networks (SNNs), a framework that utilizes simplicial complexes to capture higher-order interactions, such as multi-author collaborations. Using a massive paper-author bipartite graph derived from Semantic Scholar—comprising over 39 million papers and 12 million authors—we demonstrate how SNNs leverage the Simplicial Laplacian to perform convolutions on higher-dimensional data structures. Our results show that SNNs effectively impute missing citation data, significantly outperforming standard baselines and exhibiting strong filter transferability across different collaboration networks.

Introduction: The Limitation of Pairwise Modeling

While Graph Neural Networks (GNNs) have revolutionized relational data processing, they fundamentally rely on graphs that model interactions between pairs of nodes. In many real-world scenarios, interactions involve groups of three or more entities. For instance, a research paper co-authored by three people is more than just three separate pairs of collaborators; it represents a single cohesive unit of work. While hypergraphs can represent these sets, they often lack a rigorous topological framework to understand the underlying "shape" of the data. Simplicial Neural Networks (SNNs) address this by operating on simplicial complexes—mathematical structures that naturally encode multi-way interactions like triangles and tetrahedra.

Methods: Convolutions on Simplicial Complexes

The core of the SNN framework is the generalization of graph convolutions to higher dimensions using the spectral properties of the simplicial complex.

  1. Simplicial Complexes: Data is represented using $p$-simplices: vertices (0-simplices), edges (1-simplices), and triangles (2-simplices).
  2. Simplicial Laplacians ($L_i$): Similar to the graph Laplacian, the higher-order Simplicial Laplacian captures connectivity between different dimensions, allowing information to flow between edges and the triangles they bound.
  3. Spectral Convolution: Convolutions are defined in the frequency domain using eigenvectors of the Simplicial Laplacian. By using low-degree polynomials, the operation remains local and computationally efficient.
Simplicial complex representation of coauthorship data. (a) shows the raw data, (b) the geometric representation where a 3-author paper is a 2-simplex (triangle), and (c) the corresponding matrix representation.

Dataset and Task

The model was evaluated on Coauthorship Complexes (CC1 and CC2) derived from the Semantic Scholar Open Research Corpus. This dataset contains 39,219,709 papers and 12,862,455 authors, forming 139,268,795 edges. In this representation, authors are vertices, and a paper with $k$ authors is represented as a $(k-1)$-simplex. The primary task is missing data imputation: predicting citation counts for specific collaborations based on partial information.

Key Results

SNNs were tested on their ability to impute missing citation data at rates ranging from 10% to 50%.

  • Performance and Accuracy: SNNs significantly outperformed simple statistical baselines like the "Global Median" or "Neighbors Mean." As shown in the performance metrics, the model maintains high accuracy across different dimensions even as the percentage of missing data increases.
  • Error Distribution: The absolute error distribution reveals that the vast majority of SNN predictions are highly accurate, with errors concentrated near zero.
  • Transferability: One of the most striking findings was that filters learned on one complex (CC2) performed nearly as well on another (CC1), suggesting that SNNs capture universal patterns in how information flows through collaboration structures.
Performance analysis of SNNs. The left panels show accuracy vs. percentage of missing data across three dimensions. The right panels display the error distribution, showing most errors are near zero.

Discussion and Future Directions

While SNNs provide a principled topological approach to higher-order data, several challenges remain. Future work should focus on benchmarking against state-of-the-art hypergraph methods and developing layers that simultaneously process multiple dimensions (vertices, edges, and triangles). Additionally, scaling these complexes for even larger datasets through efficient pooling or coarsening operations will be critical for broader application in fields like fluid dynamics or global climate modeling.

Summary updated

Abstract

Paper-author bipartite graph created from Semantic Scholar's Open Research Corpus, version 2018-05-03. Vertices are papers (39,219,709 of them) in one part and authors (12,862,455 of them) in the other. A paper is connected to all its co-authors, and an author is connected to all the papers they wrote, leading to 139,268,795 edges. A citation count (the number of times the paper was cited) is available for each paper (from 0 to 37,230 citations per paper).
BibTeX
@misc{ebli2020paper,
  author = {Stefania Ebli and Michaël Defferrard and Gard Spreemann},
  title = {{Paper-author bipartite graph from Semantic Scholar}},
  year = {2020},
  eprint = {2010.03633},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.03633},
}
Figure