Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing

Chao Lou, Songlin Yang, Kewei Tu

Abstract

Nested Named Entity Recognition (NER) involves identifying entities that may be contained within other entities, a task that standard flat NER models struggle with. This paper proposes a novel approach by framing nested NER as a latent lexicalized constituency parsing problem. By integrating entity "heads"—the most linguistically significant words in a phrase—into a two-stage global inference framework, the proposed model achieves state-of-the-art performance on major benchmarks like ACE2004, ACE2005, and NNE, while maintaining high computational efficiency through GPU-parallelized algorithms.

Introduction

In traditional "flat" NER, entities are treated as non-overlapping spans. However, natural language frequently exhibits nested structures. For instance, in the phrase "University of California," the span "California" is a STATE entity nested within the larger ORGANIZATION entity "University of California."

Existing methods for nested NER typically rely on layer-based heuristics or span-based parsing. Layer-based models are susceptible to error propagation, where a failure to detect an inner entity compromises the detection of the outer one. Conversely, while span-based models allow for global inference, they often overlook the importance of entity heads—crucial linguistic anchors that signal entity types (e.g., "University" as a head for an educational organization). This work seeks to bridge this gap by combining global parsing with headword lexicalization.

Methodology

The core innovation of this research is the treatment of nested NER as Latent Lexicalized Constituency Parsing.

Lexicalized Trees and Latent Structure

Unlike standard constituency trees that only define phrase boundaries, lexicalized trees associate every phrase with a specific headword. This approach effectively merges the strengths of constituency parsing (grouping) and dependency parsing (relational linking). Since full grammatical trees are not provided in NER datasets, the model treats the non-entity portions of the tree as "latent" or hidden. It employs a masked version of the Eisner-Satta algorithm to sum over all valid tree structures that encompass the known entities.

An example of a lexicalized constituency tree representing nested entities.

Two-Stage Inference and Head Regularization

To manage the $O(N^2)$ possible spans in a sentence of length $N$, the authors implement a two-stage strategy:

  1. Stage I (Structure): The model predicts the most probable unlabeled tree structure, filtering out non-entity spans.
  2. Stage II (Labeling): Entity types (e.g., PER, GPE) are assigned to the identified spans.

Additionally, a "head regularization" technique using KL divergence is introduced to enforce the linguistic intuition that a single headword should generally govern only one entity span.

Experimental Results

The model was evaluated against four major benchmarks: ACE2004, ACE2005, GENIA, and NNE.

Performance Benchmarks

The proposed method achieved state-of-the-art (SOTA) F1 scores on ACE2004 (87.90), ACE2005 (86.91), and NNE (94.64), while remaining highly competitive on the GENIA dataset (78.44).

Performance comparison of the proposed model against existing methods on ACE and GENIA benchmarks.

Key Insights

  • The Power of Heads: Lexicalization provides a consistent boost. Identifying a word like "California" as a head significantly simplifies the boundary detection for the STATE entity.
  • Efficiency: Although the Eisner-Satta algorithm has a theoretical complexity of $O(N^4)$, the authors utilized a parallelized $O(N^3)$ implementation optimized for GPUs. This allows the model to outperform several heuristic-based models in terms of inference speed.

Conclusion

By reimagining nested NER as a parsing task enriched with lexical heads, this work provides a robust framework for handling complex entity structures. The two-stage approach effectively addresses the span imbalance problem, and the use of latent structures allows the model to learn from partially labeled data. Future work may extend this framework to joint entity and relation extraction, further leveraging the structural relationships between headwords.

Summary updated

Abstract

Nested named entity recognition (NER) has been receiving increasing attention. Recently, (Fu et al, 2021) adapt a span-based constituency parser to tackle nested NER. They treat nested entities as partially-observed constituency trees and propose the masked inside algorithm for partial marginalization. However, their method cannot leverage entity heads, which have been shown useful in entity mention detection and entity typing. In this work, we resort to more expressive structures, lexicalized constituency trees in which constituents are annotated by headwords, to model nested entities. We leverage the Eisner-Satta algorithm to perform partial marginalization and inference efficiently. In addition, we propose to use (1) a two-stage strategy (2) a head regularization loss and (3) a head-aware labeling loss in order to enhance the performance. We make a thorough ablation study to investigate the functionality of each component. Experimentally, our method achieves the state-of-the-art performance on ACE2004, ACE2005 and NNE, and competitive performance on GENIA, and meanwhile has a fast inference speed.
BibTeX
@inproceedings{lou2022nested,
  author = {Chao Lou and Songlin Yang and Kewei Tu},
  title = {{Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=nMdgQrzkmW},
}
Figure