Learning Representations of Higher-Order Relational Data

This direction is split into other directions: higher-order interaction modeling (higher-order relationship modeling). Its last version stays readable and citable here.

Version 1 of 1102 papers102 conceptsfrozen 2026-10-04

Traditional graph neural networks (GNNs) operate on a fundamental assumption: relationships are dyadic. In a standard graph, an edge connects exactly two nodes. While this abstraction is powerful, it is often a poor fit for the real world. As Shota Saito and Hideyuki Suzuki point out in Hypergraph p-Laplacian: A Differential Geometry View, relationships like a research paper with five authors or a chemical reaction with four molecules are inherently "group" dynamics.

Decomposing these into a collection of pairwise edges—a "clique" expansion—loses the structural essence of the interaction. If three people collaborate on one paper, it is a single high-order event; representing it as three separate pairwise friendships suggests a different social reality. To capture these nuances, researchers are moving toward Topological Deep Learning, a framework that replaces simple graphs with hypergraphs, simplicial complexes, and cellular sheaves.

Beyond Pairwise Edges

The most direct generalization of a graph is the hypergraph, where a "hyperedge" can contain any number of vertices. This allows for the natural representation of multinomial relations. For instance, Nozomi Nori and colleagues in Multinomial Relation Prediction in Social Data demonstrate that social interactions often involve a triplet of a user, a content creator, and a URL.

However, simply defining these sets isn't enough; we need ways to learn from them. HyperSAGE, introduced by Devanshu Arya and others, provides an inductive framework for this. Unlike methods that collapse hypergraphs into graphs, HyperSAGE uses a two-stage message-passing scheme: first aggregating information within the hyperedge, and then passing that aggregate to the constituent nodes. This preserves the "group" context during the learning process.

The Geometry of Simplicial Complexes

While hypergraphs are flexible, they lack the rigid hierarchical structure often needed for physical modeling. Simplicial complexes fill this gap by building models out of simplices: nodes (0-simplices), edges (1-simplices), triangles (2-simplices), and so on.

In Principled Simplicial Neural Networks for Trajectory Prediction, T Mitchell Roddenberry and his team treat trajectories as "flows" over these complexes. By using simplicial structures, they can enforce admissibility—ensuring the model remains invariant to how we label nodes or orient edges. This is crucial for modeling physical systems, such as fluid dynamics or traffic flow, where the "shape" of the path matters as much as the nodes visited.

To process these structures, Maosheng Yang and Elvin Isufi propose Hodge-Aware Convolutional Learning. They leverage the Hodge-Laplacian, a mathematical operator that generalizes the standard graph Laplacian. This allows the network to distinguish between different types of flows (e.g., gradient flows vs. solenoidal flows), providing a much richer vocabulary for describing how information moves through a complex system.

Cellular Sheaves and Local Consistency

One persistent problem in GNNs is oversmoothing, where node features become indistinguishable after a few rounds of message passing. Cristian Bodnar and Francesco Di Giovanni address this in Neural Sheaf Diffusion by introducing cellular sheaves.

In a standard GNN, nodes "agree" to smooth their features toward their neighbors. A sheaf, however, assigns a vector space to each node and a linear map (a "restriction map") to each edge. This allows the network to learn a custom geometry where nodes can "disagree." If two nodes are connected but belong to different classes (heterophily), the sheaf can learn a map that prevents their features from merging. Iulia Duta and colleagues extended this to Sheaf Hypergraph Networks, showing that these learnable linear maps provide a powerful inductive bias for modeling complex, multi-node interactions without losing signal to oversmoothing.

Scaling Higher-Order Learning

The primary critique of higher-order models is their computational cost. Processing every triangle or tetrahedron in a large dataset is significantly more expensive than processing edges.

Recent work has focused on making these models practical:

  • The Cellular Transformer: Rubén Ballester and Tolga Birdal introduced the Cellular Transformer, which generalizes the attention mechanism to operate on cell complexes. This allows the model to attend to higher-order "faces" (like the rings in a molecule) directly, achieving state-of-the-art results without the need for expensive graph rewiring.
  • HOPSE: To avoid the iterative cost of message passing, the HOPSE framework decomposes higher-order structures into simpler neighborhood graphs and applies positional encodings. This approach yields up to 7x speedups, making topological learning feasible for large-scale molecular benchmarks.

By moving from dyadic edges to higher-order topological structures, these researchers are building a more expressive language for deep learning—one that can finally account for the "shape" of group interactions in the real world.

Papers in this version

Showing 60 of 102 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_learning_representations_of_higher_order_a3727232,
  title = {Learning Representations of Higher-Order Relational Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learning-representations-of-higher-order-relational-data/rdr_a37272329d5e4797bf95b13477a745c3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_learning_representations_of_higher_order_a3727232_v1,
  title = {Learning Representations of Higher-Order Relational Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learning-representations-of-higher-order-relational-data/rdr_a37272329d5e4797bf95b13477a745c3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}