LIREx: Augmenting Language Inference with Relevant Explanations

Xinyan Zhao, V. G. Vinod VydiswaranAAAI 2021 · DOI

Introduction

Natural Language Inference (NLI) is a fundamental task in artificial intelligence where a model determines the logical relationship between a "premise" and a "hypothesis"—typically categorized as entailment, neutral, or contradiction. While recent advancements in large language models have improved performance, these models often lack transparency in their reasoning. Incorporating Natural Language Explanations (NLEs) aims to bridge this gap, yet existing NLE-augmented models suffer from two critical flaws: a lack of diverse rationales and the generation of "spurious" explanations that hallucinate facts to justify incorrect labels.

To address these challenges, we present LIREx (Language Inference with Relevant Explanations), a framework designed to ensure that explanations are both factually grounded and logically consistent with the input text.

The LIREx Framework

The core innovation of LIREx is its modular, four-stage pipeline that decouples explanation generation from final label prediction to prevent the model from "cheating" by inventing logic for a pre-assigned label.

The LIREx architecture pipeline, illustrating the flow from label-aware rationalization to final inference.
  1. Label-aware Rationalizer: This component identifies "rationale tokens" within the hypothesis that specifically support a potential label. For instance, it might highlight "car" when evaluating a contradiction against a premise mentioning a "motorbike."
  2. NLE Generator: Utilizing a fine-tuned GPT-2 model, LIREx generates explanations based on these rationales. Crucially, the target label is hidden during this phase, forcing the model to explain the relationship between the highlighted words and the premise objectively.
  3. Instance Selector: The system generates candidate explanations for all three possible labels. A RoBERTa-based selector then evaluates these candidates to pick the most plausible explanation given the original text.
  4. Inference Model: The final classification is performed by a model that integrates the original premise-hypothesis pair with the single best explanation identified by the selector.

Experimental Evaluation

The LIREx framework was evaluated using the Stanford Natural Language Inference (SNLI) dataset (570k pairs) and its explanation-enriched counterpart, e-SNLI. To test the model's ability to generalize beyond its training distribution, the researchers also utilized the MultiNLI dataset for out-of-domain evaluation. This tests if the model actually learned to reason or just memorized patterns specific to the SNLI dataset.

Key Results and Discussion

LIREx demonstrates significant improvements in both accuracy and robustness. On the SNLI test set, the model achieved a state-of-the-art accuracy of 91.87%, matching or exceeding the performance of top-tier models like SemBERT and NILE.

Performance comparison on the SNLI dataset, showing LIREx achieving state-of-the-art results.

Beyond raw accuracy, LIREx showed superior generalization. When tested on the MultiNLI dataset without additional fine-tuning, LIREx significantly outperformed previous models. This suggests that the framework learns genuine reasoning patterns rather than relying on dataset-specific heuristics. Furthermore, human evaluations confirmed the high quality of the generated text, with judges finding LIREx's explanations relevant to the identified rationales 95-99% of the time, compared to only 84% for previous state-of-the-art models.

Conclusion

LIREx represents a significant step toward more interpretable and robust NLI models. By anchoring explanations to specific rationales and employing a selection mechanism, the framework mitigates the risks of hallucination and spurious logic. Future work will explore extending this multi-stage approach to more complex commonsense reasoning and multi-hop question-answering tasks.

Summary updated

BibTeX
@inproceedings{zhao2021lirex,
  author = {Xinyan Zhao and V. G. Vinod Vydiswaran},
  title = {{LIREx: Augmenting Language Inference with Relevant Explanations}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i16.17708},
}
Figure