Simplicial Representation Learning with Neural $k$-Forms

Kelly Maggs, Celia Hacker, Bastian Rieck

Geometric Deep Learning (GDL) has largely been dominated by Message Passing Neural Networks (MPNNs), which aggregate information from a node's immediate neighbors. However, MPNNs suffer from well-documented limitations: over-smoothing (node features become indistinguishable after many layers), over-squashing (information bottlenecks from large neighborhoods), and difficulty capturing long-range dependencies without complex graph rewiring.

Furthermore, MPNNs often treat geometric embeddings—like the 3D coordinates of a molecule or point cloud—as just another node feature, ignoring the rich spatial structure they provide. To address this, researchers have introduced Neural $k$-forms, a novel framework that leverages differential geometry to "read" the shape of data directly, completely bypassing the standard message-passing paradigm.

The Neural $k$-Form Framework

The core innovation of this paper is the application of differential $k$-forms to neural networks. In mathematics, a $k$-form is a tool used to calculate the "volume" over a $k$-dimensional shape (e.g., a 1-form integrates over an edge or path, a 2-form over a triangle or surface).

The authors propose learning these forms using a Multi-Layer Perceptron (MLP) to model their "scaling functions." Essentially, the neural network learns how to weight different regions and orientations of the embedding space.

The Integration Matrix

Instead of iteratively passing messages between nodes, the model integrates these neural $k$-forms over the simplices of the data. This global operation produces an integration matrix where each entry represents the "signed volume" of a specific part of the graph relative to a learned feature map.

The integration of paths (1-simplices) over learned vector fields (1-forms) yields an integration matrix of signed features.

Because integration is defined over the ambient space, the model captures geometric consistency naturally. The authors also provide a Universal Approximation Theorem, proving that these neural $k$-forms can approximate any continuous, compactly supported differential form.

Key Insights and Advantages

  • Interpretability through Visualization: Because $k$-forms have explicit geometric meaning, they can be visualized. For example, a learned 1-form is essentially a vector field. In path classification experiments, the model learns vector fields that "flow" along the specific paths it is trying to identify, providing a clear, visual "why" behind its predictions.
  • Orientation Awareness: Unlike standard Graph Neural Networks (GNNs), this method naturally handles the orientation of edges and surfaces. Integration results change sign if the direction is flipped, allowing the model to capture "flow" or "circulation" within the data.
  • Parameter Efficiency: The method is remarkably lightweight, often achieving better results than traditional GNNs while using an order of magnitude fewer parameters.

Performance and Results

The researchers evaluated the Neural $k$-form (NkF) model on a mix of synthetic and real-world benchmarks, including the TU Dataset (small graphs) and MoleculeNet (large graphs).

  • MoleculeNet Superiority: On the BACE dataset (predicting molecular properties), NkF achieved an 83.50 AUROC using only 9K parameters. For comparison, a Graph Attention Network (GAT) achieved only 69.52 AUROC despite having 135K parameters.
  • Synthetic Success: In surface classification tasks, the model learned to perfectly separate different sinusoidal embeddings in 3D space—a task that is notoriously difficult for standard topological methods.
  • Structural Accuracy: On the "Letter-high" dataset, NkF maintained competitive accuracy while other models experienced significant performance drops.

Limitations and Future Directions

While highly promising, the framework has a few limitations. Primarily, it exhibits coordinate dependency; the data must be embedded in $\mathbb{R}^n$, meaning it cannot operate on purely "abstract" graphs without an initial embedding step. Additionally, integrating over very high-dimensional simplices in high-dimensional spaces could lead to computational scaling challenges.

Currently, the model does not explicitly handle equivariance (features changing predictably when the coordinate system is rotated or translated), though the authors suggest a convolutional extension. Future work will explore how these forms can capture even longer-range dependencies and apply the framework to complex regression tasks in physics and biology.

Summary updated

Abstract

Geometric deep learning extends deep learning to incorporate information about the geometry and topology data, especially in complex domains like graphs. Despite the popularity of message passing in this field, it has limitations such as the need for graph rewiring, ambiguity in interpreting data, and over-smoothing. In this paper, we take a different approach, focusing on leveraging geometric information from simplicial complexes embedded in $\mathbb{R}^n$ using node coordinates. We use differential $k$-forms in $\mathbb{R}^n$ to create representations of simplices, offering interpretability and geometric consistency without message passing. This approach also enables us to apply differential geometry tools and achieve universal approximation. Our method is efficient, versatile, and applicable to various input complexes, including graphs, simplicial complexes, and cell complexes. It outperforms existing message passing neural networks in harnessing information from geometrical graphs with node features serving as coordinates.
BibTeX
@inproceedings{maggs2024simplicial,
  author = {Kelly Maggs and Celia Hacker and Bastian Rieck},
  title = {{Simplicial Representation Learning with Neural $k$-Forms}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=Djw0XhjHZb},
}
Figure