Simple Augmentations of Logical Rules for Neuro-Symbolic Knowledge Graph Completion

Ananjan Nandi, Navdeep Kaur, Parag Singla, Mausam

Abstract

High-quality and high-coverage rule sets are imperative to the success of Neuro-Symbolic Knowledge Graph Completion (NS-KGC) models, because they form the basis of all symbolic inferences. Recent literature builds neural models for generating rule sets, however, preliminary experiments show that they struggle with maintaining high coverage. In this work, we suggest three simple augmentations to existing rule sets: (1) transforming rules to their abductive forms, (2) generating equivalent rules that use inverse forms of constituent relations and (3) random walks that propose new rules. Finally, we prune potentially low quality rules. Experiments over four datasets and five ruleset-baseline settings suggest that these simple augmentations consistently improve results, and obtain up to 7.1 pt MRR and 8.5 pt Hits@1 gains over using rules without augmentations.

Introduction

Knowledge Graphs (KGs) such as Freebase or WordNet are vital for many AI applications but are notoriously incomplete. Knowledge Graph Completion (KGC) aims to predict missing links within these massive graphs. While pure embedding models like RotatE are highly accurate, they function as "black boxes." Neuro-symbolic (NS-KGC) models attempt to combine the scalability of deep learning with the interpretability of first-order logic rules.

However, a significant bottleneck for NS-KGC is rule coverage. Current rule-generation models often produce a small set of high-quality rules that apply to only a tiny fraction of possible queries. When no rule "fires" for a given query, the model effectively ignores its symbolic component and reverts to a standard embedding model. This paper introduces a pipeline called AUG to strengthen symbolic inference by expanding ruleset coverage without drowning the model in low-quality noise.

The AUG Pipeline

The authors propose three main strategies to expand an existing ruleset (e.g., one generated by RNNLogic):

  1. Abduction: Inspired by abductive reasoning (finding the best explanation for an observation), deductive rules ($A \land B \Rightarrow C$) are transformed into abductive counterparts like $C \land B \Rightarrow A$. While not always logically certain, these provide the model with more "hypotheses" to test.
  2. Rule Inversion: Every rule is supplemented with an equivalent version using inverse relations. For example, a path $X \to Y \to Z$ is rewritten using inverse relations as $Z \to Y \to X$.
  3. Random Walk Augmentation: Local random walks on the graph propose entirely new rules, which are then filtered using a PCA Score (a metric of rule confidence).
  4. Rule Filtering (FIL): To manage the increased ruleset size and training time, PCA scores and grounding counts are used to prune low-quality or rare rules. This can reduce the augmented ruleset by up to 70% while maintaining performance.

Experimental Results

The researchers evaluated the AUG pipeline on four standard KGC benchmarks: WN18RR, FB15K-237, Kinship, and UMLS. The augmentations provided consistent gains across all datasets and multiple base models, including RNNLogic and ExpressGNN.

Main results across four datasets showing consistent improvements with rule augmentations.

As shown in the results, the augmentations led to significant accuracy boosts. On the Kinship dataset, the authors observed an increase of 7.1 points in MRR and 8.5 points in Hits@1 when comparing the original ruleset to the augmented version. These results demonstrate that in a neuro-symbolic framework, "quantity has a quality of its own." Even if abductive rules are inherently less precise, the neural component (the parameter learner) acts as a filter, learning to assign lower weights to noisy rules while benefiting from the increased coverage.

An ablation study further confirms the contribution of each component in the AUG pipeline. Removing abduction (ABD) or inversion (INV) leads to a noticeable drop in performance, highlighting that a multi-faceted approach to rule augmentation is most effective for achieving state-of-the-art results.

Ablation study showing the impact of removing individual augmentation components on WN18RR and Kinship datasets.

Key Insights and Conclusion

  • Coverage vs. Precision: While augmentations like abduction introduce less precise rules, the neuro-symbolic framework is robust enough to handle this noise, prioritizing coverage to ensure symbolic components are utilized more frequently.
  • Complementary Strengths: Rule augmentation helps symbolic models capture information that pure embedding models might miss, leading to better overall ensemble performance.
  • Efficiency through Filtering: While raw augmentation increases training time by 5-6x, PCA-based filtering effectively prunes the ruleset, keeping the computational overhead manageable (typically a 2-3x increase over baseline) with no loss in accuracy.

In conclusion, simple rule augmentations offer a powerful and interpretable way to boost the performance of neuro-symbolic KGC models. Future work may involve integrating these techniques directly into the iterative learning process of rule generators rather than applying them as a post-processing step.

Summary updated

Abstract

High-quality and high-coverage rule sets are imperative to the success of Neuro-Symbolic Knowledge Graph Completion (NS-KGC) models, because they form the basis of all symbolic inferences. Recent literature builds neural models for generating rule sets, however, preliminary experiments show that they struggle with maintaining high coverage. In this work, we suggest three simple augmentations to existing rule sets: (1) transforming rules to their abductive forms, (2) generating equivalent rules that use inverse forms of constituent relations and (3) random walks that propose new rules. Finally, we prune potentially low quality rules. Experiments over four datasets and five ruleset-baseline settings suggest that these simple augmentations consistently improve results, and obtain up to 7.1 pt MRR and 8.5 pt Hits@1 gains over using rules without augmentations.
BibTeX
@inproceedings{nandi2023simple,
  author = {Ananjan Nandi and Navdeep Kaur and Parag Singla and Mausam},
  title = {{Simple Augmentations of Logical Rules for Neuro-Symbolic Knowledge Graph Completion}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=vHaMDyhJ6k},
}
Figure