TL;DR
This paper introduces the Context Path Model (CPM), a novel approach to Knowledge Base Completion (KBC) that not only predicts missing facts but also provides human-understandable explanations for these predictions. By leveraging paths of related facts within a knowledge base, CPM offers a more transparent and verifiable way to complete knowledge graphs, addressing a key limitation of current black-box KBC models.
What Problem Does This Paper Address?
Knowledge bases (KBs) are vast repositories of structured information, but they are often incomplete. Knowledge Base Completion (KBC) aims to fill these gaps by inferring missing facts. While modern KBC methods, particularly those based on representation learning (like TransE), are effective at predicting facts, they typically operate as "black boxes." This means they don't explain why they predict a certain fact, making it difficult for humans to trust or verify their outputs. This lack of explainability is a significant hurdle for the practical adoption of KBC systems in sensitive applications. The paper tackles this by developing a method that generates explanations for KBC predictions, making them more interpretable.
Methods
The core innovation is the Context Path Model (CPM). It acts as a wrapper around existing KBC models (specifically, it's evaluated with TransE). The CPM's key idea is to use context paths – sequences of related facts surrounding a target triple – as evidence to score the correctness of that triple.
Here's a breakdown of the CPM's components:
- Context Paths: These are sequences of entities and relations that connect the two entities of a target triple. For example, if the target triple is (Theresa May, nationality, Britain), a context path might be (Theresa May, born in, Eastbourne, contained in, Britain).
- Path Correctness Score (c(p)): The CPM first learns a score for the correctness of individual paths. This is inspired by existing KBC models that can score paths, adapted to output a probability in [0,1].
- Path Relevance Score (p(t, p)): Crucially, the CPM learns how relevant a given context path
pis for assessing the correctness of a target triplet. This is a novel component, learned using a translation-based approach similar to TransE. - Triple Correctness Score (c(t, Pt)): The final score for a triple
tis a weighted average of the correctness scores of its context pathsPt, where the weights are determined by the relevance scores. This means paths that are highly relevant and likely correct strongly influence the triple's score. - Explanation Generation: The paths with the highest relevance scores are presented as explanations. If a path has a high relevance and a high correctness score, it supports the triple; if it has high relevance but a low correctness score, it argues against the triple.
The training process is split into two steps: first, training the path correctness scores, and then training the path relevance scores. This separation ensures that path correctness is interpretable independently of its relevance to a specific triple.
Datasets
The primary dataset used for training and evaluation is FB15K, a subset of the Freebase knowledge base. The paper also mentions using the FB15K dataset extracted by Bordes et al. (2013). No new benchmarks were introduced; the focus was on evaluating the CPM on an existing, widely used KBC dataset.
Key Insights and Ideas
- Paths as Explanations: The central insight is that paths of related facts can serve as intuitive and verifiable explanations for KBC predictions. This moves beyond simple predictions to offer transparency.
- Context Matters: The CPM demonstrates that the surrounding context (represented by paths) is a valuable source of information for assessing the correctness of a fact, even when the fact itself is known.
- Relevance is Key: Learning the relevance of a path to a specific triple is crucial. A path might be factually correct but irrelevant to the triple in question, and vice-versa. The CPM explicitly models this relationship.
- Bridging Symbolic and Sub-symbolic: The CPM attempts to bridge the gap between symbolic reasoning (paths and logical connections) and sub-symbolic representation learning (embeddings), offering a hybrid approach.
- Type-Matched Corruption for Negatives: The authors found that using "type-matched corruption" for generating negative samples (incorrect facts) leads to more meaningful evaluation results for explainable KBC, as it forces models to capture finer-grained semantics.
Key Results
- Performance Trade-off: The CPM's performance on fact prediction (measured by H@10 and MQ) is slightly lower than the best-performing TransE model (edge-trained), with a drop of around 6-11% for H@10. However, the authors argue this is a reasonable trade-off for the added explainability.
- Path Correctness: The CPM performs well on predicting the correctness of longer paths (lengths 2 and 3), especially when trained with a path-based regimen, validating the model's core design.
- Meaningful Explanations: A qualitative analysis (annotation study) showed that a significant portion of the paths identified by the CPM as relevant were either logically equivalent to the triple or provided probable evidence for its correctness. About half of the annotated pairs were "equivalent," and another quarter were "probable."
- CPM\t (No Triple Info): A variant of CPM that excluded information about the target triple itself performed significantly worse, highlighting that the triple's own semantics are still the primary driver of prediction accuracy.
Limitations and Future Work
- TransE Dependency: The current study primarily uses TransE as the underlying KBC model. Future work should explore the CPM's effectiveness with other composable KBC models like Bilinear-diag.
- Path Selection Heuristics: The CPM uses heuristics to limit the number of context paths considered (e.g., limited length, filtering redundant paths). Future research could explore more sophisticated or relaxed path selection methods.
- Symmetric Correlation: The current CPM primarily captures symmetric correlations (where the path's correctness is directly tied to the triple's). Future work aims to generalize the model to asymmetrical relations, enabling the capture of necessary or sufficient conditions for a fact.
- Broader Applications: The authors suggest that the CPM's output could be useful for structural analysis of KBs, such as identifying equivalent relations or improving KB consistency.
Summary updated