Combining Spans into Entities: A Neural Two-Stage Approach for Recognizing Discontiguous Entities

Bailin Wang, Wei Lu

TL;DR

This paper introduces a novel two-stage neural network approach to identify complex entities in text that can be broken into non-contiguous parts and even overlap with other entities. By breaking down the problem into segment extraction and segment merging, the model effectively handles these challenging structures without the ambiguity issues of previous methods, achieving state-of-the-art results.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP) that aims to identify and categorize entities (like people, organizations, or medical conditions) within text. Traditionally, NER systems assume entities are contiguous sequences of words and do not overlap. However, in real-world scenarios, especially in specialized domains like medical texts, entities can be "discontiguous" (e.g., "laceration ... esophagus") and "overlapping" (e.g., "blood in stomach" and "stomach ... lac"). These complex structures create a vast search space for possible entity combinations, making them difficult for existing models to recognize accurately. Previous attempts to address this often relied on complex tagging schemes that introduced ambiguity during decoding or used heuristics to resolve it.

Methods

The authors propose a neural two-stage approach to tackle discontiguous and overlapping entities. This approach decomposes the problem into two distinct subtasks:

  1. Segment Extraction: The first stage focuses on identifying all potential "segments" within a sentence. These segments can either be complete entities themselves or parts of larger discontiguous entities. The paper leverages a neural segmental hypergraph model, which is adept at representing and encoding all possible overlapping segment combinations without ambiguity. This model uses LSTMs to capture word-level and span-level representations.
  2. Segment Merging: The second stage takes the extracted segments and learns to combine them into complete discontiguous entities. A binary classifier is used to determine if a combination of segments forms a valid entity. To capture interactions between segments within a potential entity, another LSTM is employed.

A key aspect of their method is the joint learning of these two stages using a shared text encoder. This allows information to flow between the segment extraction and merging processes, leading to improved performance.

Datasets

The paper evaluates its model on a subset of the original data from the SemEval-2014 task and the ShARe/CLEF eHealth Evaluation Lab (SHEL) 2013. This subset was specifically chosen to include sentences with at least one discontiguous entity, as the original datasets had a low percentage of such cases. The authors note that in their chosen subset, 53.6% of entities are discontiguous, and overlapping entities also frequently appear.

Key Insights and Ideas

  • Decomposition is Key: Breaking down the complex problem of recognizing discontiguous and overlapping entities into two simpler, sequential subtasks (segment extraction and segment merging) effectively circumvents the ambiguity issues faced by single-stage models.
  • Hypergraphs for Overlap: Neural segmental hypergraphs provide a powerful, unambiguous way to represent all possible overlapping segment combinations, which is crucial for the first stage of their model.
  • Shared Encoder Benefits: Jointly training the segment extraction and merging modules with a shared text encoder significantly improves performance, suggesting that these two tasks are not entirely independent and can benefit from shared representations.
  • No External Features Needed: The proposed neural model achieves state-of-the-art results even without relying on handcrafted features (like POS tags or word clusters) that were common in previous methods. This highlights the power of end-to-end neural learning.

Key Results

The paper reports significant improvements over existing baselines in both non-neural and neural settings.

  • In the neural setting, their model achieved the best F₁ score of 53.7% (compared to the best baseline of 52.8% for Graph (all)).
  • Even when compared to non-neural models that used handcrafted features, their neural model outperformed them.
  • The authors also demonstrated the benefit of using a shared encoder, showing a performance gain from 54.0% to 56.1% F₁ score in one comparison.
  • When assessing the handling of multiple entity types, their model again achieved the best F₁ score.

Limitations and Future Work

The authors acknowledge a potential limitation related to exposure bias during training. Specifically, the segment merging classifier is trained only on correct segments. This means it might not learn to recover from errors made by the segment extraction stage during decoding.

Future work could explore ways to expose the segment merging classifier to incorrect segments during training to mitigate this issue. Additionally, while their model is intrinsically capable of handling multiple entity types, they suggest further analysis in this area.

Summary updated

BibTeX
@inproceedings{wang2019combining,
  author = {Bailin Wang and Wei Lu},
  title = {{Combining Spans into Entities: A Neural Two-Stage Approach for Recognizing Discontiguous Entities}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=Ze0iJF0yQ1A},
}
Figure