Multi-turn Inference Matching Network for Natural Language Inference

Chunhua Liu, Shan Jiang, Hainan Yu, Dong YuLecture Notes in Computer Science 2018 · DOI · arXiv · OpenAlex

Abstract

Natural Language Inference (NLI) traditionally relies on "one-pass" matching-aggregation frameworks that may lose nuanced information. We present the Multi-turn Inference Matching Network (MIMN), which employs an iterative reasoning process. By decomposing matching features into joint, difference, and similarity perspectives and processing them through a multi-turn inference layer with a memory component, MIMN achieves state-of-the-art performance on complex NLI benchmarks, particularly those requiring multi-premise reasoning.

Introduction

Natural Language Inference (NLI) is a fundamental task in natural language processing, requiring a model to determine if a premise entails, contradicts, or is neutral toward a hypothesis. Conventional approaches typically follow a "matching-aggregation" paradigm: word-level alignments are computed, matching features are extracted and concatenated, and a single classification pass is performed.

However, this "one-pass" approach suffers from several limitations. First, concatenating diverse features into a single high-dimensional vector can lead to significant information loss. Second, it lacks the ability to model complex interactions between different types of evidence, such as how a word overlap (similarity) might be negated by a specific word difference. Finally, it treats all features with equal priority, failing to mimic human reasoning which often prioritizes specific cues (like contradictions) during inference.

Architecture: The Multi-turn Inference Matching Network

The proposed MIMN architecture addresses these flaws by introducing an iterative refinement process. The model consists of four primary layers: Encoding, Attention, Matching, and the novel Multi-turn Inference Layer.

Architecture of the Multi-turn Inference Matching Network (MIMN).

Multi-Perspective Matching

Instead of a single feature vector, the Matching Layer generates three distinct sequences to capture different logical relationships:

  • Joint-feature ($u^c$): A concatenation of premise and hypothesis vectors to preserve comprehensive information.
  • Diff-feature ($u^s$): Element-wise subtraction to highlight semantic differences.
  • Sim-feature ($u^m$): Element-wise multiplication to emphasize similarities.

Iterative Inference and Memory

The core innovation is the Multi-turn Inference Layer. The model performs three sequential turns of inference, with each turn focusing on one of the matching features (Joint → Diff → Sim). This order reflects a heuristic approach: first grasping the overall context, then identifying discrepancies, and finally confirming commonalities.

To maintain coherence across turns, a Memory Component (similar to a GRU or LSTM cell state) stores the state of reasoning. An update gate regulates the flow of information, ensuring that insights from previous turns inform the current step. This allows the model to build a cumulative "inference message" rather than treating each perspective in isolation.

Experimental Setup

The MIMN model was evaluated on three diverse NLI benchmarks: the large-scale SNLI dataset, the science-focused SciTail dataset, and the Multiple Premise Entailment (MPE) dataset.

Statistics of the datasets used for evaluation.

The MPE dataset is particularly challenging as it requires evaluating a hypothesis against four distinct premises, necessitating more sophisticated evidence aggregation than standard single-premise tasks.

Results and Discussion

MIMN demonstrated strong performance across all benchmarks, with its most significant gains appearing in complex reasoning scenarios.

Performance Highlights

  • SNLI: The model achieved 88.3% accuracy (single model) and 89.3% (ensemble), placing it among the top-performing models at the time of publication.
  • MPE: MIMN achieved a remarkable 9.7% absolute improvement over previous state-of-the-art models, reaching 66.0% accuracy. This confirms the effectiveness of the multi-turn mechanism in filtering and aggregating evidence across multiple sentences.
  • SciTail: The model reached 84.0% accuracy, outperforming the CAFE model by 0.7%.

The Role of Memory and Iteration

Ablation studies highlighted the necessity of the memory component. Removing it (MIMN-memory) resulted in a 0.8% drop in SNLI accuracy, indicating that the interaction between reasoning turns is critical. The iterative approach allows the model to refine its understanding, effectively "thinking" through the problem in stages rather than attempting a single, complex calculation.

Conclusion

The Multi-turn Inference Matching Network demonstrates that iterative reasoning is a powerful paradigm for NLI. By isolating different matching perspectives and linking them via a memory component, the model better captures the nuances of logical relationships. Future work may explore dynamic turn ordering or scaling the number of turns to further enhance reasoning depth. While computationally more intensive than one-pass models, the significant accuracy gains—especially on multi-premise tasks—justify the multi-turn approach.

Summary updated

BibTeX
@inproceedings{liu2018multi,
  author = {Chunhua Liu and Shan Jiang and Hainan Yu and Dong Yu},
  title = {{Multi-turn Inference Matching Network for Natural Language Inference}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2018},
  doi = {10.1007/978-3-319-99501-4\_11},
}
Figure