Knowledge Graphs (KGs) serve as structured repositories of human knowledge, representing facts as triples of (subject, relation, object). Because real-world KGs are notoriously incomplete, researchers widely employ Knowledge Graph Embedding (KGE) models to predict missing links. By mapping discrete entities and relations into continuous, high-dimensional vector spaces, KGEs achieve remarkable predictive accuracy.
However, this geometric success comes at a cost. As highlighted by Russa Biswas et al. in their comprehensive survey, Knowledge Graph Embeddings: Open Challenges and Opportunities, these models function as uninterpretable black boxes. When a model projects entities into a 500-dimensional latent space and computes a high similarity score for a new link, it cannot explain why that link is plausible. This lack of transparency limits their deployment in high-stakes domains where human verification is non-negotiable, such as causal event forecasting (Sola S. Shirai et al., Rule-Based Link Prediction over Event-Related Causal Knowledge in Wikidata) or automated medical recommendations (Edoardo Ramalli et al., Demystifying Drug Repurposing Domain Comprehension with Knowledge Graph Embedding).
To bridge this gap, a prominent research direction focuses on making link prediction interpretable. The core intuition is to connect continuous latent spaces back to discrete, human-readable evidence—either by extracting post-hoc explanations from black-box models or by designing models that are interpretable by design.
Post-Hoc Explanations: Finding the Evidence Subgraph
One approach to interpretability preserves the high predictive power of deep embedding models but constructs an independent explanation layer on top of them. If a black-box model makes a correct prediction, what specific training data or structural patterns drove that decision?
Influential Examples
One way to explain a prediction is to trace it back to the training instances that most heavily influenced the model's parameters. Adrianna Janik et al. introduced this concept to KGs with ExamplE (Explaining Link Predictions in Knowledge Graph Embedding Models with Influential Examples). By identifying "prototype" training facts that reside near the predicted link within the model's latent space, the framework provides a human-readable, graph-based justification of the model's internal state.
Synergistic Paths and Ontological Rules
Instead of isolated training facts, explanations can take the form of multi-hop paths within the graph. Ye Sun et al. developed eXpath (eXpath: Explaining Knowledge Graph Link Prediction with Ontological Closed Path Rules), which searches the neighborhood of a prediction to extract semantically meaningful relation paths and ontological rules.
Often, a single path is insufficient to justify a complex relationship. To address this, Tengfei Ma et al. proposed KGExplainer (Towards Synergistic Path-based Explanations for Knowledge Graph Completion: Exploration and Evaluation), which uses a perturbation-based greedy search to discover "synergistic paths"—multiple interconnected reasoning chains that work in tandem to support a predicted fact.
[Subject Entity] ──(Relation A)──> [Intermediate Entity] ──(Relation B)──> [Object Entity]
│ ▲
└───────────────────────────(Predicted Relation)─────────────────────────┘
By isolating these subgraphs, post-hoc methods reveal the underlying structural evidence that aligns with the model's continuous mathematical calculations.
Inductive Rule Learning: Mining Symbolic Logic
A more radical solution to the interpretability problem bypasses black-box embeddings entirely, replacing them with symbolic logic. As noted by Meng Qu et al. in RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs, there is a historical trade-off between the high performance of embeddings and the transparency of logic. Rule-based models solve this by automatically mining Horn clauses (e.g., $B(x, y) \land C(y, z) \Rightarrow A(x, z)$) directly from the graph.
Optimization and Type Awareness
To make rule discovery efficient and compact, Sanjeeb Dash et al. framed rule selection as a constrained optimization problem using linear programming (Rule Induction in Knowledge Graphs Using Linear Programming). To ensure the learned rules make semantic sense, Hong Wu et al. developed TyRuLe (Learning Typed Rules over Knowledge Graphs), an architecture that explicitly integrates entity types (such as Person or City) into the path-exploration phase, preventing the model from generating logically valid but nonsensical rules.
Temporal and Sequential Reasoning
Graphs are often dynamic, requiring models to reason across time. Yunpu Ma et al. addressed this with TLogic (TLogic: Temporal Logical Rules for Explainable Link Forecasting on Temporal Knowledge Graphs), which extracts temporal random walks and lifts them into abstract, time-consistent rules. For long-range reasoning chains, Shengyao Lu et al. introduced R5 (R5: Rule Discovery with Reinforced and Recurrent Relational Reasoning), which uses reinforcement learning to treat rule composition as a sequential decision-making process.
Because these rules are written in discrete symbolic logic, they are inherently interpretable; the execution of the rule is the explanation.
Neural-Symbolic Co-Design: Interpretable by Design
The frontier of this research direction moves away from the dichotomy of "post-hoc explanations vs. pure symbolic rules" toward neural-symbolic integration. These models embed logical constraints and symbolic structures directly into neural architectures, achieving the predictive robustness of embeddings alongside the auditability of logic.
Fuzzy Logic and Proof Trees
Instead of mapping queries to single vector points, neural-symbolic models map them to logical operations. Mikhail Galkin et al. demonstrated this with GNN-QE (Neural-Symbolic Models for Logical Queries on Knowledge Graphs), combining Graph Neural Networks with fuzzy logic operators. This allows the system to handle missing data and complex multi-hop queries while permitting users to inspect every intermediate reasoning step.
Continuous Graph Space Symbolic Execution Space
[ Neural Embeddings ] ──(Fuzzy Logic)──> [ Multi-Hop Proof Tree ]
(Handles noise/gaps) (Auditable, step-by-step)
Taking this design philosophy further, Ehsan Bonabi Mobaraki et al. introduced a framework (Interpretable Link Prediction via Neural-Symbolic Reasoning) that outputs explicit, multi-hop logical proof trees alongside each prediction. If the model predicts a link, it simultaneously outputs the exact mathematical and logical derivation tree that led to that conclusion.
Shared Concepts and Instance Prototypes
Other architectures achieve inherent interpretability by structuring how information is transferred or grouped. For instance, Michael Qizhe Shieh et al. developed ITransF (An Interpretable Knowledge Transfer Model for Knowledge Base Completion), which uses a sparse attention mechanism to map relations to shared, human-interpretable "hidden concepts."
Alternatively, Wanyun Cui et al. proposed an instance-based learning paradigm (Instance-based Learning for Knowledge Base Completion), demonstrating that many successful rule-based models actually operate by identifying analogical equivalencies between specific "prototype" entities.
By forcing the neural network to route its calculations through discrete concepts, logical gates, or structural prototypes, these models eliminate the black-box problem at the architectural level. The continuous vector space handles the messiness, noise, and sparsity of real-world graphs, while the symbolic framework guarantees that the final output can be unraveled into a clear, step-by-step logical justification.
Key Foundational Work
Anytime Bottom-Up Rule Learning for Knowledge Graph Completion (2019) — contributed 1 concept elements to this direction.
Graph Pattern Entity Ranking Model for Knowledge Graph Completion (2019) — contributed 1 concept elements to this direction.
Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications (2019) — contributed 1 concept elements to this direction.