HetSAGE: Heterogenous Graph Neural Network for Relational Learning (Student Abstract)

Vince Jankovics, Michaël Garcia Ortiz, Eduardo AlonsoAAAI 2021 · DOI

Abstract

Relational learning, the task of modeling complex dependencies between entities, has historically been dominated by Neuro-Symbolic Learning (NSL). However, traditional NSL methods often fail to scale or preserve the inherent topology of relational data. This paper proposes HetSAGE, a framework that bridges logic-based reasoning and deep learning by representing logic programs as heterogeneous graphs. By leveraging Graph Neural Networks (GNNs), HetSAGE achieves superior performance on standard relational benchmarks, demonstrating that GNNs represent a more efficient and scalable evolution of neuro-symbolic methods.

Introduction

Relational learning—understanding how different entities like actors, movies, and directors relate to one another—has traditionally been the domain of Neuro-Symbolic Learning (NSL). NSL attempts to combine the "brain" of neural networks with the "logic" of symbolic reasoning.

However, traditional NSL methods, such as CILP++, often rely on a process called propositionalisation. This involves flattening complex relational databases into simple attribute-value tables. This approach is brittle: it struggles to scale, cannot handle an arbitrary number of connections, and often loses the rich structural information inherent in the data. The authors argue that Graph Neural Networks (GNNs) are naturally better suited for this task but have lacked rigorous comparison to NSL standards until now.

The HetSAGE Framework

The authors propose a two-step pipeline to modernize relational learning by transforming logic-based problems into a graph-centric deep learning task.

  1. Logic-to-Graph Transformation: Logic programs (sets of "if-then" rules) are transformed into grounded graphs. In this setup, terms become nodes, attributes become node features, and relationships become labeled edges.
  2. The HetSAGE Architecture: To process these graphs, HetSAGE combines several state-of-the-art GNN techniques:
  3. Sampling & Aggregation: Borrowed from GraphSAGE, this allows the model to handle large graphs by sampling fixed-size neighborhoods.
  4. Heterogeneous Embedding: Inspired by HetGNN, this maps different types of nodes (e.g., a "paper" node vs. an "author" node) into a shared mathematical space.
  5. Edge Feature Handling: Similar to MPNNs, the model considers the specific type of relationship (edge) when passing messages between nodes.
  6. Label Propagation: A key feature where the model can "see" the labels of neighboring nodes (excluding the target node) to inform its predictions.
Visualization of the HetSAGE neighborhood aggregation process across a heterogeneous graph.

Experimental Evaluation

The model was evaluated on three classic relational datasets to test its versatility:

  • MUTA188: A dataset of 188 molecules where the goal is to predict mutagenicity.
  • CORA: A citation network of scientific papers. The authors tested several variations: binary vs. multi-class, and with/without text features ("words") or neighbor labels.
  • MovieLens: A movie recommendation and classification dataset involving movies, actors, and directors.

Results and Key Insights

HetSAGE outperformed the state-of-the-art NSL model (CILP++) across all benchmarks. The performance gap was most significant in the CORA citation task, where HetSAGE's ability to exploit graph structure led to a ~22% lead over the symbolic baseline.

Comparative performance of CILP++ and GNN (HetSAGE) on relational learning benchmarks.

Several key insights emerged from the study:

  • GNNs as "NSL 2.0": The authors argue that GNNs are the structural evolution of NSL, preserving data topology in a way that traditional symbolic learners cannot.
  • The Power of Neighborhood Labels: Including the labels of neighboring nodes (label propagation) provided a massive performance boost.
  • Structure Over Features: In the CORA dataset, the model performed better when text content was removed, provided it still had access to neighbor labels. This suggests that in relational tasks, structural signals ("who you know") can be stronger than individual features ("what you say").
  • Logic as Pre-processing: Instead of using logic inside the neural network, the authors suggest using logic programming to enrich the graph database before GNN processing.

Conclusion and Future Work

As a student abstract, this work establishes a foundation for GNNs as a superior alternative to traditional NSL. Future research directions include testing on much larger-scale datasets to further demonstrate scalability, performing more exhaustive hyperparameter optimization, and exploring the integration of "common sense" logic into the graph construction phase to further enhance predictive power.

Summary updated

BibTeX
@inproceedings{jankovics2021hetsage,
  author = {Vince Jankovics and Michaël Garcia Ortiz and Eduardo Alonso},
  title = {{HetSAGE: Heterogenous Graph Neural Network for Relational Learning (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17898},
}
Figure