Communicative Message Passing for Inductive Relation Reasoning

Shuangjia Zheng

Introduction

Knowledge Graphs (KGs) serve as the backbone for numerous AI applications, from search engines to recommendation systems. However, these graphs are often incomplete. While traditional link prediction models excel in transductive settings—where all entities are known during training—they struggle with inductive reasoning. Inductive reasoning requires a model to predict relationships between entities that were never seen during the training phase, a common scenario in dynamic real-world environments.

The Challenge of Inductive Reasoning

Existing inductive models, such as GraIL, typically rely on extracting local subgraphs around target entities. However, these approaches suffer from two primary limitations:

  1. Loss of Directionality: Many models treat subgraphs as undirected, which obscures the logic of asymmetric relations (e.g., "Parent of" vs. "Child of").
  2. Static Relation Modeling: Relations are often treated as simple weights for node updates rather than dynamic features that can evolve through interaction with the graph structure.

Methodology: The CoMPILE Framework

To address these gaps, the authors propose CoMPILE (Communicative Message Passing neural network for Inductive reLation rEasoning). CoMPILE introduces a "communicative" kernel that allows node and edge embeddings to iteratively refine each other.

Directed Subgraph Extraction

Unlike previous methods that might include irrelevant neighbors, CoMPILE focuses on the directed enclosing subgraph between a target head and tail entity. This approach preserves the logical flow of information and reduces computational overhead by filtering out noise.

Comparison of different subgraph extraction strategies, highlighting the importance of directed enclosing subgraphs.

Communicative Message Passing

The core innovation of CoMPILE is its message-passing scheme. In standard Graph Neural Networks (GNNs), edges typically act as static conduits for node information. In CoMPILE, node embeddings are updated based on incoming edges, and simultaneously, edge embeddings are updated based on the nodes they connect. This bidirectional communication ensures that relational information is deeply integrated into the entity representations.

The CoMPILE architecture illustrating the communicative flow between node (N), relation (R), and edge (E) representations.

Scoring and Attention

The model employs an enhanced edge attention mechanism that considers the entire target triplet (head, relation, tail). For the final prediction, it uses a TransE-style scoring function ($h + r \approx t$), which is inherently well-suited for capturing the directionality of asymmetric and anti-symmetric relations.

Experimental Evaluation

CoMPILE was benchmarked against inductive versions of three standard datasets: WN18RR (WordNet), FB15k-237 (Freebase), and Nell-995. The researchers also introduced "Post-processed Datasets" to ensure a more rigorous evaluation by filtering out "empty" subgraphs where no path exists between the target entities.

Results and Discussion

CoMPILE achieved state-of-the-art results across nearly all benchmarks. Notably, on the WN18RR dataset, CoMPILE reached an AUC-PR of 99.80%, significantly outperforming GraIL's 92.72%.

A key finding was the model's robustness to entity swapping. In tests involving asymmetric relations, CoMPILE maintained high accuracy when head and tail entities were exchanged, whereas baseline models saw a sharp decline. This confirms that CoMPILE successfully learns the underlying structural logic of the graph rather than relying on entity identities.

Conclusion

CoMPILE represents a significant step forward for inductive reasoning in Knowledge Graphs. By treating relations as dynamic, first-class citizens and preserving graph directionality, it provides a robust framework for reasoning over unseen entities. Future work may explore "relation-only" message passing to further decouple reasoning from specific node coordinates, potentially leading to even more scalable and abstract inductive models.

Summary updated

Abstract

Relation prediction for knowledge graphs aims at predicting missing relationships between entities. Despite the importance of inductive relation prediction, most previous works are limited to a transductive setting and cannot process previously unseen entities. The recent proposed subgraph-based relation reasoning models provided alternatives to predict links from the subgraph structure surrounding a candidate triplet inductively. However, we observe that these methods often neglect the directed nature of the extracted subgraph and weaken the role of relation information in the subgraph modeling. As a result, they fail to effectively handle the asymmetric/anti-symmetric triplets and produce insufficient embeddings for the target triplets. To this end, we introduce a Communicative Message Passing neural network for Inductive reLation rEasoning, CoMPILE, that reasons over local directed subgraph structures and has a vigorous inductive bias to process entity-independent semantic relations. In contrast to existing models, CoMPILE strengthens the message interactions between edges and entitles through a communicative kernel and enables a sufficient flow of relation information. Moreover, we demonstrate that CoMPILE can naturally handle asymmetric/anti-symmetric relations without the need for explosively increasing the number of model parameters by extracting the directed enclosing subgraphs. Extensive experiments show substantial performance gains in comparison to state-of-the-art methods on commonly used benchmark datasets with variant inductive settings.
BibTeX
@inproceedings{zheng2021communicative,
  author = {Shuangjia Zheng},
  title = {{Communicative Message Passing for Inductive Relation Reasoning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i5.16554},
}
Figure