Abstract
This paper introduces a composite approach to Recognizing Textual Entailment (RTE) that utilizes a routing mechanism to select between syntactic and semantic solvers. By leveraging Definition Knowledge Graphs (DKGs) built from structured dictionaries, the system achieves superior performance on benchmark datasets while providing human-readable justifications for its entailment decisions.
Introduction
Recognizing Textual Entailment (RTE)—determining if a "Text" (T) logically implies a "Hypothesis" (H)—is fundamental to natural language understanding. Entailment ranges from purely syntactic transformations (e.g., active to passive voice) to complex semantic leaps requiring external world knowledge (e.g., knowing a "spike" in violence is an "increase"). While deep learning models offer high accuracy, they often function as "black boxes." This research addresses the need for flexible, explainable RTE systems by bridging the gap between syntactic precision and semantic depth.
Methodology
The core of the proposed system is a routing mechanism that analyzes word overlap between T and H to determine the appropriate processing path.
Syntactic and Semantic Solvers
- The Router: Sentence pairs with total or null overlap are directed to the Syntactic Model. Pairs with partial overlap, indicating semantic substitutions, are sent to the Semantic Model.
- Syntactic Model (Tree Edit Distance): This component uses the APTED algorithm to calculate the cost of transforming the dependency tree of T into H. Low transformation costs confirm entailment.
- Semantic Model (Graph Navigation): This novel component utilizes DKGs derived from WordNet, Wiktionary, and Wikipedia. A "Distributional Navigation Algorithm" searches for paths between differing terms in the graph.
- Interpretability Module: If a path is found, the system converts graph nodes into natural language justifications. For example, a path linking "spike" to "increase" generates a human-readable explanation based on lexical relations.
Experimental Results
The composite approach was evaluated on the BPI (Princeton Boeing-ISI) and GHS (Guardian Headlines Sample) datasets.
- BPI Dataset: The Composite (WordNet) model achieved an F-measure of 0.66, significantly outperforming the standard Edit Distance baseline (0.53) and Maximum Entropy classifiers (0.51).
- GHS Dataset: The model reached an F-measure of 0.60, compared to 0.45 for the syntactic baseline, demonstrating robustness on "in-the-wild" news data.
- Knowledge Base Comparison: Surprisingly, WordNet outperformed the massive Wikipedia graph. The authors found that everyday language concepts in standard dictionaries are more effective for entailment than the noisy named-entity data found in encyclopedias.
Discussion and Conclusion
The routing mechanism prevents the model from "over-thinking" simple grammatical variations, which often leads to false positives in purely semantic models. Furthermore, the use of dictionary-based graphs allows the system to generate justifications that feel human-like because they are derived from how humans define words.
Limitations include a reliance on the quality of structured definitions; for instance, Wikipedia's lack of verb definitions hindered its performance. Future research will focus on integrating these interpretable knowledge graphs into modern deep learning architectures to combine transparency with the raw predictive power of neural networks.
Summary updated