A Bi-LSTM-RNN Model for Relation Classification Using Low-Cost Sequence Features

Fei Li, Meishan Zhang, Guohong Fu, Tao Qian, Donghong JiarXiv 2016 · arXiv

TL;DR

This paper introduces a Bi-LSTM-RNN model for relation classification that achieves high performance using "low-cost" sequence features (like words and POS tags) instead of expensive structural features like dependency trees. By segmenting sentences into five distinct parts, the authors demonstrate that the text between two entities is a powerful proxy for structural information, often matching the performance of much more complex models.

What Problem Does This Paper Address?

Relation classification—identifying the relationship between two entities in a text (e.g., "The burst was caused by pressure" indicates a Cause-Effect relation)—is a cornerstone of information extraction.

Historically, the best-performing models relied on structural features, such as dependency parsers that map out the grammatical "skeleton" of a sentence. However, these features come with two major drawbacks:

  1. High Computational Cost: Parsing every sentence is slow and resource-intensive.
  2. Domain Sensitivity: A parser trained on news articles often fails when applied to specialized fields like biology or medicine, hurting the model's accuracy.

The authors set out to prove that a deep learning model can achieve comparable results using only sequence-based features that are easier to compute and more adaptable across different domains.

Methods

The authors propose a Bidirectional Long Short-Term Memory Recurrent Neural Network (Bi-LSTM-RNN). The core of their approach is how they represent the input:

  • Five-Part Segmentation: Instead of treating the sentence as one long string, the model divides it into five segments based on the target entities: before context, former entity, middle context, latter entity, and after context.
  • Feature Fusion: The model combines pre-trained word embeddings, character-level embeddings (to handle out-of-vocabulary words), and linguistic tags like POS and WordNet hypernyms.
  • Pooling: After processing the sequence with the Bi-LSTM, the model applies four types of pooling (max, min, average, and standard deviation) to each of the five segments. These pooled representations are concatenated and fed into a final softmax layer.
The Bi-LSTM-RNN architecture showing the five-part segmentation and pooling layers.

Key Insights and Ideas

  • The "Middle" is King: The authors found that the middle context (the words between the two entities) is the most critical feature. In the SemEval dataset, using only the middle context yielded an F1-score of 81.1%, nearly as high as the full model.
  • A Proxy for Structure: One of the most interesting findings is that the middle context acts as a "low-cost" replacement for the Shortest Dependency Path (SDP). Statistical analysis showed that 82% of the words found in a formal dependency path also appear in the simple sequence of words between the entities.
Statistical overlap between the middle context and the Shortest Dependency Path (SDP).
  • Domain Independence: Because the model doesn't rely on a rigid syntactic parser, it transitions more effectively to the biomedical domain, where traditional parsers often struggle.

Key Results

  • SemEval-2010: The model achieved an F1-score of 82.0% using only sequence features. When the authors experimentally added dependency features, it rose to 83.1%, placing it on par with state-of-the-art models that are much more computationally expensive.
  • BioNLP-ST 2016: The model obtained the third-best result in the official evaluation. Notably, it performed better when the "context window" was expanded to include two continuous sentences, helping it capture relations that traditional single-sentence models would miss.
  • Ablation Studies: The results showed that WordNet hypernyms were the most effective auxiliary feature, boosting the F1-score by about 1%.

Limitations and Future Work

  • Noise in the Context: While the middle context captures the necessary structural information, it also includes "noise"—words that aren't relevant to the relation but happen to be between the entities. Future work may focus on methods to automatically filter this noise.
  • Cross-Sentence Complexity: While the model showed promise on the BioNLP task, identifying relations across multiple sentences remains significantly harder than within a single sentence, likely due to the increased amount of irrelevant text as the window grows.

Summary updated

Abstract

Relation classification is associated with many potential applications in the artificial intelligence area. Recent approaches usually leverage neural networks based on structure features such as syntactic or dependency features to solve this problem. However, high-cost structure features make such approaches inconvenient to be directly used. In addition, structure features are probably domain-dependent. Therefore, this paper proposes a bi-directional long-short-term-memory recurrent-neural-network (Bi-LSTM-RNN) model based on low-cost sequence features to address relation classification. This model divides a sentence or text segment into five parts, namely two target entities and their three contexts. It learns the representations of entities and their contexts, and uses them to classify relations. We evaluate our model on two standard benchmark datasets in different domains, namely SemEval-2010 Task 8 and BioNLP-ST 2016 Task BB3. In the former dataset, our model achieves comparable performance compared with other models using sequence features. In the latter dataset, our model obtains the third best results compared with other models in the official evaluation. Moreover, we find that the context between two target entities plays the most important role in relation classification. Furthermore, statistic experiments show that the context between two target entities can be used as an approximate replacement of the shortest dependency path when dependency parsing is not used.
BibTeX
@misc{li2016bi,
  author = {Fei Li and Meishan Zhang and Guohong Fu and Tao Qian and Donghong Ji},
  title = {{A Bi-LSTM-RNN Model for Relation Classification Using Low-Cost Sequence Features}},
  year = {2016},
  eprint = {1608.07720},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1608.07720},
}
Figure