Multi-Perspective Inferrer: Reasoning Sentences Relationship from Holistic Perspective

Zhen Cheng, Zaixiang Zheng, Xin-Yu Dai, Shujian Huang, Jiajun ChenarXiv 2019 · arXiv

Abstract

Natural Language Inference (NLI) is a fundamental task in natural language processing that requires determining the logical relationship between a premise and a hypothesis. Traditional models often rely heavily on alignment mechanisms, which can lead to a "Neutral Trap" where high word overlap incorrectly triggers an "Entailment" prediction. We present the Multi-Perspective Inferrer (MPI), a novel framework that utilizes dynamic routing to categorize sentence segments into specific logical perspectives: Entailment, Neutral, or Contradiction. By moving beyond simple word-matching to a holistic reasoning approach, our Injection MPI (IMPI) variant achieves state-of-the-art performance on the SNLI and MultiNLI datasets, demonstrating particular robustness in scenarios with low word overlap.

Introduction

Determining whether a hypothesis is true (entailment), false (contradiction), or undetermined (neutral) given a premise is the core of Natural Language Inference. While modern deep learning models have made significant strides, they often rely on a "crutch" of word alignment. This reliance creates a significant failure mode known as the "Neutral Trap." For instance, if a premise is "The cat is on the mat" and the hypothesis is "The black cat is on the mat," an alignment-heavy model may predict "Entailment" due to the high word overlap, ignoring the fact that the color "black" is an unverified, neutral detail.

To address this, we propose that NLI requires a holistic view where different segments of the sentence contribute unequally to different logical conclusions. Our Multi-Perspective Inferrer (MPI) treats sentence tokens as "parts" and the three logical relations as "wholes," using dynamic routing to assign importance.

Methodology

Routing-by-Agreement

The core innovation of the MPI is the use of Dynamic Routing, a concept adapted from Capsule Networks. Instead of simple max-pooling or attention-based aggregation, the model employs an iterative "Routing-by-Agreement" process. This mechanism assigns token-level features to one of three high-level capsules: Entailment, Neutral, or Contradiction.

MPI Variants

We explore two primary architectural variants of the MPI:

  1. Snoop MPI: In this symmetric version, premise and hypothesis tokens are routed individually. However, they are allowed to "snoop" on each other’s high-level capsules to find mutual agreement during the routing process.
  2. Injection MPI (IMPI): Recognizing that NLI is inherently asymmetric—the hypothesis is judged relative to the premise—this version injects premise information directly into the hypothesis routing process. This allows the model to treat the premise as the necessary context for interpreting the hypothesis.
The architectural diagrams for Snoop Multi-Perspective Inferrer and Injection Multi-Perspective Inferrer.

Auxiliary Supervision

To ensure that the capsules learn semantically meaningful features (e.g., the "Entailment" capsule actually captures entailment-related signals), we introduce an auxiliary supervised loss signal. This constraint makes the model's internal logic highly interpretable and ensures the routing mechanism aligns with human logical categories.

Experimental Results

Performance on Standard Benchmarks

We evaluated the MPI as a modular component plugged into both BiLSTM-based models and BERT. On the Stanford Natural Language Inference (SNLI) dataset, BERT + IMPI achieved a state-of-the-art accuracy of 90.8%. On the Multi-Genre NLI (MultiNLI) dataset, it reached 85.2% (matched) and 84.1% (mismatched), outperforming standard BERT baselines and more complex specialized architectures.

Robustness in the "Disaster Area"

The authors identify the "disaster area" of NLI as sentence pairs with very low word overlap (less than 20%). In these cases, standard alignment-based models lose their primary signal and often fail. The MPI significantly reduced error rates in these scenarios, particularly for the Neutral and Entailment classes, by capturing the significance of unaligned parts that contribute to a neutral or contradictory relationship.

Interpretability and Visualization

One of the most compelling aspects of the MPI is its inherent interpretability. By visualizing the routing weights, we can observe how the model "points" to specific words to justify its classification.

Routing visualization demonstrating how the model assigns specific words in the hypothesis to Entailment (EN), Neutral (NE), or Contradiction (CON) capsules.

As shown in the visualization, for a neutral pair, the MPI correctly routes the "new" or unverified information in the hypothesis to the Neutral capsule, while shared information is routed toward Entailment. This confirms that the model is performing logical reasoning rather than simple pattern matching.

Conclusion

The Multi-Perspective Inferrer offers a robust solution to the limitations of alignment-heavy NLI models. By utilizing dynamic routing to categorize information into logical perspectives, the MPI achieves superior performance and provides a transparent look into its decision-making process. Future work will explore adapting this routing-by-agreement policy to other sentence-pair tasks, such as machine reading comprehension and semantic matching.

Summary updated

Abstract

Natural Language Inference (NLI) aims to determine the logic relationships (i.e., entailment, neutral and contradiction) between a pair of premise and hypothesis. Recently, the alignment mechanism effectively helps NLI by capturing the aligned parts (i.e., the similar segments) in the sentence pairs, which imply the perspective of entailment and contradiction. However, these aligned parts will sometimes mislead the judgment of neutral relations. Intuitively, NLI should rely more on multiple perspectives to form a holistic view to eliminate bias. In this paper, we propose the Multi-Perspective Inferrer (MPI), a novel NLI model that reasons relationships from multiple perspectives associated with the three relationships. The MPI determines the perspectives of different parts of the sentences via a routing-by-agreement policy and makes the final decision from a holistic view. Additionally, we introduce an auxiliary supervised signal to ensure the MPI to learn the expected perspectives. Experiments on SNLI and MultiNLI show that 1) the MPI achieves substantial improvements on the base model, which verifies the motivation of multi-perspective inference; 2) visualized evidence verifies that the MPI learns highly interpretable perspectives as expected; 3) more importantly, the MPI is architecture-free and compatible with the powerful BERT.
BibTeX
@misc{cheng2019multi,
  author = {Zhen Cheng and Zaixiang Zheng and Xin-Yu Dai and Shujian Huang and Jiajun Chen},
  title = {{Multi-Perspective Inferrer: Reasoning Sentences Relationship from Holistic Perspective}},
  year = {2019},
  eprint = {1911.03668},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.03668},
}
Figure