Abstract
Natural Language Inference (NLI) is a fundamental task in natural language processing. While deep learning models have achieved impressive results, they often lack the common-sense lexical knowledge that humans use for reasoning. This paper introduces the Knowledge-based Inference Model (KIM), which enriches the state-of-the-art Enhanced Sequential Inference Model (ESIM) with external lexical relations from WordNet. KIM achieves state-of-the-art performance on the SNLI dataset and demonstrates significantly better generalization and robustness, particularly in low-data regimes and on challenge datasets designed to test lexical understanding.
Introduction
Natural Language Inference (NLI) involves determining whether a premise sentence entails, contradicts, or is neutral toward a hypothesis. Modern neural models typically learn these relationships from scratch (tabula rasa) using large-scale datasets like SNLI. However, these models often fail when encountering word pairs not seen during training, as they lack general world knowledge. For instance, a model might struggle to identify a contradiction between "wheat" and "corn" if their relationship isn't explicitly captured in the training distribution.
The Knowledge-based Inference Model (KIM)
To address this, the authors propose KIM, which integrates structured lexical knowledge into the neural architecture. KIM builds upon the ESIM framework by introducing three knowledge-enhanced components:
- Knowledge-Enriched Co-Attention: This component modifies the standard attention mechanism by incorporating a "knowledge bonus." If two words share a semantic relationship in WordNet, their attention score is boosted, helping the model align related concepts across sentences.
- Local Inference Collection with External Knowledge: When comparing aligned word pairs, KIM concatenates a "knowledge vector" to the standard representation. This vector explicitly encodes five WordNet relations: Synonymy, Antonymy, Hypernymy, Hyponymy, and Co-hyponymy.
- Knowledge-Enhanced Inference Composition: During the final aggregation of word-level information, KIM employs a weighted pooling mechanism that prioritizes words with strong external semantic links.
Experimental Results
The authors evaluated KIM on the SNLI and MultiNLI benchmarks, as well as the Glockner et al. (2018) challenge set.
State-of-the-Art Performance
KIM achieved a new state-of-the-art single-model accuracy of 88.6% on the SNLI test set at the time of publication. It also showed competitive performance on MultiNLI, particularly in the "in-domain" setting.
Robustness and Generalization
A key strength of KIM is its robustness to "adversarial" word swaps. On the Glockner challenge set—where models are tested on sentences with single-word changes (e.g., "dog" to "cat")—the baseline ESIM's accuracy dropped significantly to 66%. In contrast, KIM maintained a high accuracy of 83.5%, demonstrating that external knowledge helps the model move beyond simple statistical correlations.
Low-Data Efficiency
The impact of external knowledge is most evident when training data is limited. As shown in Figure 2, KIM significantly outperforms the baseline ESIM in low-data scenarios. With only 0.8% of the training data, KIM achieves a gain of over 10% in absolute accuracy.
Ablation studies (denoted as KIM (A), (I), and (C) for the three components) reveal that the Local Inference Collection (I) is the most critical component. Providing the model with explicit relation labels (like "antonym") during the comparison phase provides the most substantial performance boost.
Conclusion
KIM demonstrates that incorporating structured lexical knowledge into neural NLI models significantly improves performance, robustness, and data efficiency. By leveraging WordNet, the model gains a form of "common sense" that allows it to generalize better than models trained solely on text corpora. Future work may involve extending this framework to include broader factual knowledge from sources like Wikidata or handling more complex phrasal and logical inferences.
Summary updated