Exploring Knowledge Graphs in an Interpretable Composite Approach for Text Entailment

Vivian Dos Santos Silva, André Freitas, Siegfried HandschuhAAAI 2019 · DOI

Abstract

This paper introduces a composite approach to Recognizing Textual Entailment (RTE) that utilizes a routing mechanism to select between syntactic and semantic solvers. By leveraging Definition Knowledge Graphs (DKGs) built from structured dictionaries, the system achieves superior performance on benchmark datasets while providing human-readable justifications for its entailment decisions.

Introduction

Recognizing Textual Entailment (RTE)—determining if a "Text" (T) logically implies a "Hypothesis" (H)—is fundamental to natural language understanding. Entailment ranges from purely syntactic transformations (e.g., active to passive voice) to complex semantic leaps requiring external world knowledge (e.g., knowing a "spike" in violence is an "increase"). While deep learning models offer high accuracy, they often function as "black boxes." This research addresses the need for flexible, explainable RTE systems by bridging the gap between syntactic precision and semantic depth.

Methodology

The core of the proposed system is a routing mechanism that analyzes word overlap between T and H to determine the appropriate processing path.

The system architecture illustrates the routing of sentence pairs to either Tree Edit Distance or Graph Navigation modules based on word overlap.

Syntactic and Semantic Solvers

  1. The Router: Sentence pairs with total or null overlap are directed to the Syntactic Model. Pairs with partial overlap, indicating semantic substitutions, are sent to the Semantic Model.
  2. Syntactic Model (Tree Edit Distance): This component uses the APTED algorithm to calculate the cost of transforming the dependency tree of T into H. Low transformation costs confirm entailment.
  3. Semantic Model (Graph Navigation): This novel component utilizes DKGs derived from WordNet, Wiktionary, and Wikipedia. A "Distributional Navigation Algorithm" searches for paths between differing terms in the graph.
  4. Interpretability Module: If a path is found, the system converts graph nodes into natural language justifications. For example, a path linking "spike" to "increase" generates a human-readable explanation based on lexical relations.
A graph navigation example showing the path from 'spike' to 'increase' through the intermediate concept 'rise', providing a basis for logical justification.

Experimental Results

The composite approach was evaluated on the BPI (Princeton Boeing-ISI) and GHS (Guardian Headlines Sample) datasets.

  • BPI Dataset: The Composite (WordNet) model achieved an F-measure of 0.66, significantly outperforming the standard Edit Distance baseline (0.53) and Maximum Entropy classifiers (0.51).
  • GHS Dataset: The model reached an F-measure of 0.60, compared to 0.45 for the syntactic baseline, demonstrating robustness on "in-the-wild" news data.
  • Knowledge Base Comparison: Surprisingly, WordNet outperformed the massive Wikipedia graph. The authors found that everyday language concepts in standard dictionaries are more effective for entailment than the noisy named-entity data found in encyclopedias.

Discussion and Conclusion

The routing mechanism prevents the model from "over-thinking" simple grammatical variations, which often leads to false positives in purely semantic models. Furthermore, the use of dictionary-based graphs allows the system to generate justifications that feel human-like because they are derived from how humans define words.

Limitations include a reliance on the quality of structured definitions; for instance, Wikipedia's lack of verb definitions hindered its performance. Future research will focus on integrating these interpretable knowledge graphs into modern deep learning architectures to combine transparency with the raw predictive power of neural networks.

Summary updated

BibTeX
@inproceedings{silva2019exploring,
  author = {Vivian Dos Santos Silva and André Freitas and Siegfried Handschuh},
  title = {{Exploring Knowledge Graphs in an Interpretable Composite Approach for Text Entailment}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33017023},
}
Figure