CIN++: Enhancing Topological Message Passing

Lorenzo Giusti, Teodora Reu, Francesco Ceccarelli, Cristian Bodnar, Pietro Liò

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable success in learning from graph-structured data. However, they face significant limitations in expressive power, struggling with long-range interactions and lacking a principled approach to modeling higher-order structures and group interactions. Cellular Isomorphism Networks (CINs) recently addressed most of these challenges with a message passing scheme based on cell complexes. Despite their advantages, CINs make use only of boundary and upper messages which do not consider a direct interaction between the rings present in the underlying complex. Accounting for these interactions might be crucial for learning representations of many real-world complex phenomena such as the dynamics of supramolecular assemblies, neural activity within the brain, and gene regulation processes. In this work, we propose CIN++, an enhancement of the topological message passing scheme introduced in CINs. Our message passing scheme accounts for the aforementioned limitations by letting the cells to receive also lower messages within each layer. By providing a more comprehensive representation of higher-order and long-range interactions, our enhanced topological message passing scheme achieves state-of-the-art results on large-scale and long-range chemistry benchmarks.

Introduction: Beyond Pairwise Interactions

Standard Graph Neural Networks (GNNs) typically treat data as a collection of nodes and edges. While effective for many tasks, this pairwise perspective suffers from two major drawbacks in complex domains like chemistry. First is oversquashing, where information loss occurs as it travels across many edges between distant nodes. Second is the inability to model group interactions naturally; for instance, aromatic rings in molecules behave as single units rather than just a set of individual bonds.

Previous work introduced Cellular Isomorphism Networks (CINs), which lift graphs into "cell complexes" by adding 2D cells (rings) to the graph structure. However, CINs were limited by a hierarchical message-passing scheme where rings could only communicate with edges, and edges with nodes. This created a bottleneck, preventing direct interaction between rings and limiting the model's ability to capture complex supramolecular dynamics.

Methods: Introducing Lower Messages

CIN++ enhances the topological message-passing scheme by introducing Lower Messages. In a cell complex, cells can communicate through three primary channels:

  1. Boundary Messages: Information flows from lower-dimensional cells to higher ones (e.g., an edge to a ring).
  2. Upper Messages: Cells of the same dimension interact if they share a higher-dimensional boundary (e.g., two nodes sharing an edge).
  3. Lower Messages: Cells of the same dimension interact if they share a lower-dimensional boundary.
Types of topological messages in CIN++.

The introduction of lower messages is the key innovation of CIN++. It allows rings to communicate directly with adjacent rings if they share a common edge, and edges to communicate if they share a node. This architecture uses Multi-Layer Perceptrons (MLPs) for message and update functions, maintaining permutation invariance. Theoretically, CIN++ is proven to be at least as expressive as the Cellular Weisfeiler-Lehman (CWL) test.

Key Insights: Topological Shortcuts and Symmetry

The inclusion of lower messages provides several advantages:

  • Direct Ring Communication: Bypassing the hierarchical "wait-and-see" approach leads to faster convergence and better representations of chemical phenomena like aromatic stacking.
  • Topological Shortcuts: Higher-order cells act as "express lanes." Instead of hopping through numerous atoms, a message can jump across a ring structure, significantly mitigating oversquashing.
  • Natural Symmetry: The model's structure (0-cells for atoms, 1-cells for bonds, 2-cells for rings) aligns with the physical reality of chemistry, providing a strong inductive bias.
Comparison of information propagation in CIN vs. CIN++ (CIN with Lower Messages).

Results and Evaluation

CIN++ was evaluated on several high-impact benchmarks, demonstrating state-of-the-art (SOTA) performance:

  • ZINC: Achieved a SOTA Mean Absolute Error (MAE) of 0.077 on the ZINC-Subset.
  • Peptides-func & Peptides-struct: On these long-range interaction benchmarks, CIN++ reached a Test AP of 0.6569 and a Test MAE of 0.2523, respectively, proving its effectiveness at capturing long-distance dependencies.
  • TUDatasets: Outperformed existing models on four out of five datasets (MUTAG, PTC_MR, PROTEINS, and NCI109).
  • OGB-MOLHIV: Showed strong performance on this large-scale molecular dataset.

Conclusion

While CIN++ introduces some computational overhead due to the "lifting" process (identifying rings), this is a one-time pre-processing step. The enhanced message-passing scheme significantly improves the model's ability to handle long-range and higher-order interactions. Future work may explore even higher-dimensional cells or more efficient lifting maps to further scale these topological benefits.

Summary updated

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable success in learning from graph-structured data. However, they face significant limitations in expressive power, struggling with long-range interactions and lacking a principled approach to modeling higher-order structures and group interactions. Cellular Isomorphism Networks (CINs) recently addressed most of these challenges with a message passing scheme based on cell complexes. Despite their advantages, CINs make use only of boundary and upper messages which do not consider a direct interaction between the rings present in the underlying complex. Accounting for these interactions might be crucial for learning representations of many real-world complex phenomena such as the dynamics of supramolecular assemblies, neural activity within the brain, and gene regulation processes. In this work, we propose CIN++, an enhancement of the topological message passing scheme introduced in CINs. Our message passing scheme accounts for the aforementioned limitations by letting the cells to receive also lower messages within each layer. By providing a more comprehensive representation of higher-order and long-range interactions, our enhanced topological message passing scheme achieves state-of-the-art results on large-scale and long-range chemistry benchmarks.
BibTeX
@inproceedings{giusti2024cin,
  author = {Lorenzo Giusti and Teodora Reu and Francesco Ceccarelli and Cristian Bodnar and Pietro Liò},
  title = {{CIN++: Enhancing Topological Message Passing}},
  booktitle = {IJCNN},
  year = {2024},
  doi = {10.1109/ijcnn60899.2024.10650343},
}
Figure