Abstract
Considerable attention has recently been devoted to the problem of automatically extending knowledge bases by applying some form of inductive reasoning. While the vast majority of existing work is centred around so-called knowledge graphs, in this paper we consider a setting where the input consists of a set of (existential) rules. To this end, we exploit a vector space representation of the considered concepts, which is partly induced from the rule base itself and partly from a pre-trained word embedding. Inspired by recent approaches to concept induction, we then model rule templates in this vector space embedding using Gaussian distributions. Unlike many existing approaches, we learn rules by directly exploiting regularities in the given rule base, and do not require that a database with concept and relation instances is given. As a result, our method can be applied to a wide variety of ontologies. We present experimental results that demonstrate the effectiveness of our method.
Introduction
Traditional Knowledge Base Completion (KBC) typically focuses on predicting missing facts within knowledge graphs, such as identifying a missing capital city for a country. However, many professional ontologies are defined primarily by rules (e.g., "All citrus fruits contain Vitamin C") rather than instance-level facts. These rule bases are often incomplete, yet traditional rule mining techniques require large databases of examples to function. This paper addresses the challenge of rule completion in the absence of instance data by framing it as a cognitive task: inducing general rules from patterns observed within the existing rule base.
Methodology
The proposed model, Rule Induction (RI), leverages two primary components: rule templates and vector space representations.
Rule Templates and Embeddings
Rule templates are generated by replacing relations in existing rules with variables. To determine which concepts satisfy these templates, the model represents relations as vectors derived from two sources:
- Word Embeddings: Pre-trained Skip-gram vectors (e.g., GloVe) that capture lexical similarities.
- Rule-Base Embeddings: Vectors obtained via Matrix Factorization (SVD/PCA) of the existing rule base, capturing structural similarities between relations.
Bayesian Concept Induction
A key innovation is the modeling of rule templates as ellipsoidal regions rather than simple spheres. In a high-dimensional vector space, not all dimensions are equally relevant for a specific rule. For instance, a rule about "citrus fruits" should be sensitive to biological genus but perhaps indifferent to color.
By modeling templates as Gaussians with diagonal covariance matrices, the model creates flexible ellipsoidal boundaries. A Bayesian approach is employed to estimate these parameters, providing robustness even when only a few examples of a rule are available, thereby preventing overfitting.
Experimental Evaluation
The RI model was evaluated against several ontologies, including large-scale systems like SUMO and OpenCyc, as well as domain-specific ones like Wine, Economy, Transport, and Vicodi. The experimental setup involved hiding existing rules and tasking the model with re-discovering them among a set of "distractor" rules.
Results
The full RI model consistently outperformed baseline methods such as AnalogySpace (AS) and simple Vector Space (VS) similarity models.
As shown in the results table, RI achieved significant improvements in precision and F1 scores. For instance, on the SUMO ontology, the model reached a Precision@10 of 0.62. The results also highlighted that combining word embeddings with rule-base embeddings yields the best performance, as they provide complementary lexical and structural information.
Discussion and Limitations
While the RI model demonstrates strong performance without requiring instance data, it currently operates under a closed-world assumption for prior probability estimation. Furthermore, the current implementation focuses on unary and binary templates, leaving more complex multi-relation structures for future work. The model's reliance on meaningful relation names for certain "restricted templates" also suggests that performance might vary on ontologies using opaque identifiers. Future research will explore more sophisticated conceptual space models to capture nuanced semantic relationships beyond Gaussian distributions.
Summary updated