Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

Bin Shang, Yinliang Zhao, Jun Liu, Di WangAAAI 2024 · DOI

TL;DR

This paper introduces MGTCA, a novel approach for knowledge graph completion (KGC) that leverages multiple geometric spaces (hyperbolic, hypersphere, and Euclidean) to enrich entity embeddings. It proposes a trainable convolutional attention network (TCAN) that can dynamically switch between different graph neural network (GNN) types, eliminating the need for manual pre-validation of local graph structures.

What Problem Does This Paper Address?

Knowledge graphs (KGs) are powerful tools for representing real-world data, but they are often incomplete. Knowledge graph completion (KGC) aims to fill these gaps by predicting missing facts. Traditional methods, including graph convolutional networks (GCNs) and graph attention networks (GATs), have shown promise but suffer from limitations. GCNs can over-aggregate neighbor information, leading to redundancy, while GATs can focus too narrowly on specific neighbors, potentially missing broader structural context. Both approaches are also sensitive to the local structure of the KG, often requiring expensive pre-validation to determine the optimal GNN type for different entities. Furthermore, existing methods primarily operate in Euclidean space, which may not fully capture the complex, hierarchical, or spherical structures inherent in many KGs.

Methods

MGTCA tackles these challenges with three core innovations:

  • Mixed Geometry Message Function (MGMF): This function generates richer neighbor messages by integrating information from hyperbolic (negative curvature), hypersphere (positive curvature), and Euclidean (zero curvature) spaces. This allows the model to capture a wider range of structural properties within the KG.
  • Trainable Convolutional Attention Network (TCAN): Instead of relying on a fixed GNN architecture, TCAN comprises multiple GNN types (GCN, GAT, and a hybrid KGCAT) within a single trainable formulation. Through trainable coefficients, each layer can autonomously learn to switch between these GNN types, adapting to the local structure of the KG without explicit pre-validation. This dynamic adaptation aims to leverage the strengths of each GNN type while mitigating their weaknesses.
  • Mixed Geometry Scoring Function (MGSF): This function calculates the plausibility of a triple (head entity, relation, tail entity) by combining a novel prediction function with a similarity function. The similarity function explicitly considers distances in hyperbolic, hypersphere, and Euclidean spaces, further enriching the model's ability to represent and score relationships across different geometric manifolds.

Datasets

The paper evaluates MGTCA on three standard benchmark datasets for knowledge graph completion:

  • FB15k-237: A subset of FB15k with inverse relations removed.
  • YAGO3-10: A subset of YAGO3 containing entities with at least 10 relations.
  • WN18RR: A subset of WN18, specifically designed to remove symmetric/antisymmetric and compositional relation patterns that can lead to trivial predictions.

Key Insights and Ideas

  • Geometry Matters: The paper strongly advocates for moving beyond Euclidean space in KGC. By incorporating hyperbolic and hypersphere geometries, MGTCA can better represent the inherent structures of KGs, leading to improved performance.
  • Dynamic GNN Adaptation: The TCAN's ability to dynamically switch between GCN, GAT, and KGCAT behaviors per layer is a significant advancement. This eliminates the manual, computationally expensive step of pre-validating graph structures and allows the model to learn the optimal aggregation strategy for each entity's neighborhood.
  • Unified Framework: MGTCA's integration of mixed geometry messaging, trainable attention, and multi-geometry scoring provides a comprehensive framework that addresses multiple limitations of prior KGC methods simultaneously.

Key Results

MGTCA consistently outperforms state-of-the-art (SOTA) models across the evaluated datasets (FB15k-237, YAGO3-10, and WN18RR). Specifically:

  • On FB15k-237, MGTCA achieves improvements of 2-3% in MRR and Hits@10 compared to SOTA results.
  • On YAGO3-10 and WN18RR, MGTCA shows significant gains in Hits@3 and Hits@10.
  • The model demonstrates superior performance compared to both Euclidean and non-Euclidean KGC approaches, including recent GNN-based models like MRGAT and GreenKGC.
  • Ablation studies confirm that each component of MGTCA (MGMF, TCAN, and MGSF) contributes positively to the overall performance. Removing any single geometric space also leads to performance degradation, highlighting the importance of the multi-geometry approach.

Limitations and Future Work

  • Computational Cost: While MGTCA offers significant improvements, the integration of multiple geometric spaces and trainable GNNs likely increases computational complexity compared to simpler models.
  • Hyperparameter Tuning: The paper mentions grid search for hyperparameter optimization, suggesting that finding optimal settings might still require considerable effort.
  • Future Work: The authors suggest that their trainable convolutional attention network (TCAN) could be further explored for other graph-based tasks. The ability to dynamically adapt GNN types per layer opens avenues for more sophisticated and adaptive graph learning models. Further research could also explore more advanced geometric spaces or novel ways to combine information across them.

Summary updated

BibTeX
@inproceedings{shang2024mixed,
  author = {Bin Shang and Yinliang Zhao and Jun Liu and Di Wang},
  title = {{Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i8.28745},
}
Figure