Abstract: Knowledge embeddings are key ingredients of advanced question-answering and recommender systems. Even though their predictions are accurate, they are rather hard to interpret by human users; interpretability techniques are needed so as to provide meaningful human-friendly explanations for prediction generated by embeddings. We propose a novel model-agnostic method inspired by local surrogate approaches that generates faithful explanations for knowledge embedding predictions.
Introduction
Knowledge Graphs (KGs) are the backbone of modern AI, powering everything from search engines to recommendation systems. However, the most powerful way to use them—Knowledge Embeddings (KEs)—is notoriously opaque. While KEs are excellent at predicting missing facts (like "Who is the spouse of X?"), they do so by mapping entities into high-dimensional vector spaces where "meaning" is just a collection of coordinates.
In their paper, Faithfully Explaining Predictions of Knowledge Embeddings, Gustavo Padilha Polleti and Fabio Gagliardi Cozman introduce a way to peek inside this black box and get human-readable reasons for why an AI made a specific prediction.
Problem Statement
Knowledge Embeddings (like TransE) represent entities and relations as vectors. They are highly accurate at "Knowledge Base Completion"—filling in the blanks in a database—but their reasoning is trapped in a "latent space" that humans can't interpret.
Existing solutions have major gaps. As shown in the comparison below, most methods fail to satisfy all three critical criteria: being model-agnostic, providing human-friendly explanations, and maintaining faithfulness to the original model's logic.
- Model-specific methods: Only work for one type of embedding (e.g., CrossE).
- Global surrogates: Try to explain the whole model at once, which often leads to "low fidelity" (the explanation doesn't actually match what the model did for a specific case). XKE, for instance, is model-agnostic and human-friendly but lacks faithfulness.
- Non-human-friendly: Some methods explain predictions using vector dimensions, which is meaningless to a non-expert.
The challenge is to create a method that is model-agnostic (works for any KE), faithful (accurately reflects the specific prediction), and human-friendly (uses words and relationships).
Proposed Methodology
The authors' approach is inspired by LIME (Local Interpretable Model-agnostic Explanations), but tailored for relational graphs. Their process follows these steps:
- Perturbation: To explain a prediction (e.g., why the model thinks "Jesus" has the religion "Judaism"), they generate a "neighborhood" of data points by slightly perturbing the head entity's vector.
- Labeling: They run these perturbed points through the original "black-box" KE model to see how the predictions change.
- Feature Extraction via the Black Box: This is the core innovation. To get interpretable features, they use the KE model itself to find "one-sided path features." For example, it asks the model: "What is the nationality of this perturbed entity?" or "What is its profession?"
- Local Surrogate Training: They train a simple, inherently interpretable model (a Sparse Logistic Regression) on these path features to mimic the black box's behavior in that local neighborhood.
- Explanation: The weights (coefficients) of the simple model become the explanation. High positive weights are "Pros" for the prediction; negative weights are "Cons."
Experimental Setup
The study uses FB13, a well-known subset of the Freebase knowledge graph.
- Stats: 75,043 entities and 13 relations.
- Content: It contains roughly 345,000 triples (facts) divided into training, validation, and testing sets.
Key Insights
- The KE as its own Translator: The most significant insight is using the black-box embedding model to extract its own interpretable features. By querying the embedding to find paths (like
spouse -> gender), the researchers bridge the gap between latent vectors and semantic meaning. - Local vs. Global: By focusing only on the "neighborhood" of a single prediction, the surrogate model can be much simpler and more accurate than a global model trying to explain the entire knowledge graph at once.
- The Value of "Cons": Unlike some previous methods that only show supporting evidence, this approach identifies features that refute a prediction. This provides a "broader understanding," showing why the model chose one answer over a close second.
Results and Discussion
The authors evaluated their method qualitatively using the TransE model.
- Meaningful Explanations: For the prediction
(Jesus, religion, Judaism), the model identified supporting reasons likeethnicity -> Jewandprofession -> Rabbi. - Coherence: The model showed that the same feature could be a "Pro" for one prediction and a "Con" for another. For example, the feature
ethnicity -> Jewsupported the prediction of "Judaism" but acted as a negative weight (a reason to deny) the prediction of "Christianity." - Fidelity: Because the surrogate is trained locally, it achieves higher fidelity than previous global methods like XKE, meaning the explanation is a truer reflection of the black box's logic.
Limitations and Future Work
- Path Complexity: In this initial study, the authors limited the path length ($L$) to 1. Real-world reasoning often requires longer chains of logic.
- Subjective Evaluation: The results currently rely on manual qualitative inspection. Future work should include evaluations with human subjects to measure how much these explanations actually help users.
- Counterfactuals: The authors suggest exploring "counterfactual" explanations—telling a user exactly what would need to change for the model to give a different answer.
Summary updated