A Sentence Interaction Network for Modeling Dependence between Sentences

Biao Liu, Minlie HuangACL 2016 · DOI

Abstract

Modeling the relationship between sentence pairs is fundamental to tasks such as answer selection and dialogue act analysis. Traditional approaches often utilize independent encoding or simple scalar similarity matrices, which lack the capacity to capture nuanced semantic influences. We propose the Sentence Interaction Network (SIN), an architecture that models "information flow" as high-dimensional vectors between word pairs. By integrating these interaction states directly into the encoding process, SIN allows words in one sentence to dynamically influence the representations of words in another. Our experiments on the WikiQA and Switchboard Dialogue Act datasets demonstrate that SIN significantly outperforms strong baselines, achieving state-of-the-art results without manual feature engineering.

Introduction

Understanding how two sentences relate—whether one answers the other or follows it in a conversation—requires more than just comparing global sentence vectors. Historically, models have either encoded sentences independently, missing context-specific nuances, or used similarity matrices that reduce complex semantic interactions to single scalar values. For instance, a question word like "who" should fundamentally change how a model perceives a person's name in a candidate answer. To address this, we argue for a mechanism where words can dynamically influence each other through rich, vector-valued interactions.

The Sentence Interaction Network (SIN)

The SIN architecture processes sentence pairs through a multi-stage pipeline designed to capture deep dependencies:

  1. Initial Encoding: Each sentence is first processed by a primary LSTM layer to generate context-aware hidden states for every word.
  2. Interaction Mechanism: For each word in the target sentence, the model computes a "candidate interaction state" with every word in the source sentence. Unlike traditional attention mechanisms that produce scalars, these states are high-dimensional vectors.
  3. Aggregation and Integration: These influences are aggregated and fed into a second LSTM layer. This ensures the final representation of each word is inherently "aware" of the entire context of the paired sentence.
The SIN architecture illustrating the interaction between two sentence sequences $s_1$ and $s_2$.

To further enhance the model's ability to handle multi-word phrases, we introduced SIN-CONV, which incorporates a Convolutional Neural Network (CNN) layer before the interaction step to capture local n-gram features.

Experimental Evaluation

We evaluated SIN on two distinct benchmarks:

  • WikiQA: A task requiring the ranking of candidate answers for a given question.
  • Switchboard Dialogue Act (SwDA): A corpus of telephone transcripts where each utterance must be classified into one of 42 dialogue acts (e.g., Statement, Agreement).

These datasets test the model's ability to handle both question-answering logic and conversational flow.

Results and Discussion

The SIN model demonstrated superior performance across both tasks. On the WikiQA dataset, the SIN-CONV variant achieved a MAP of 0.674 and an MRR of 0.693, outperforming both standard LSTMs and more complex models like the Attention-based CNN (ABCNN).

Comparison of SIN variants against baseline models on the WikiQA dataset.

Key insights from our analysis include:

  • Vector-Valued Flows: High-dimensional interaction states allow the model to encode how a word is influenced (e.g., identifying it as a specific answer type), rather than just that it is related.
  • Dynamic Influence: Visualizations confirm that the model learns to prioritize content-heavy words while ignoring stop words, effectively bridging the semantic gap between sentences.
  • Feature Extraction: The architecture achieves these results using raw word embeddings (GloVe), proving its effectiveness in automatic feature extraction.

Conclusion

The Sentence Interaction Network provides a powerful framework for modeling sentence dependencies by treating interaction as a high-dimensional flow. While the model excels at capturing word-level and local phrase interactions, future work will explore extending SIN to tree-based structures to better handle complex, long-range linguistic dependencies.

Summary updated

BibTeX
@inproceedings{liu2016sentence,
  author = {Biao Liu and Minlie Huang},
  title = {{A Sentence Interaction Network for Modeling Dependence between Sentences}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2016},
  doi = {10.18653/v1/p16-1053},
}
Figure