Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing

Chao Lou, Songlin Yang, Kewei Tu

Abstract

Nested named entity recognition (NER) has been receiving increasing attention. Recently, research has adapted span-based constituency parsers to tackle nested NER by treating nested entities as partially-observed constituency trees and proposing the masked inside algorithm for partial marginalization. However, these methods often fail to leverage entity heads, which are crucial for entity mention detection and typing. In this work, we resort to more expressive structures: lexicalized constituency trees where constituents are annotated by headwords. We leverage the Eisner-Satta algorithm to perform partial marginalization and inference efficiently. Additionally, we propose a two-stage strategy, a head regularization loss, and a head-aware labeling loss to enhance performance. Our method achieves state-of-the-art performance on ACE2004, ACE2005, and NNE, and competitive performance on GENIA, while maintaining fast inference speeds.

Introduction

Traditional NER systems are typically designed for "flat" entities—simple, non-overlapping spans like [New York City]. However, real-world language is frequently nested. For example, in the phrase "[the [University] of [California]]", "University" is an organization, "California" is a state, and the entire phrase constitutes a larger educational entity.

Existing approaches to Nested NER generally fall into two categories: layer-based/heuristic models and parsing-based models. Layer-based models recognize entities sequentially but often suffer from error propagation. Parsing-based models treat entities as nodes in a tree, allowing for global optimization, but they have historically ignored entity heads—the most semantically significant word in a phrase (e.g., "University" in the example above). This paper bridges that gap by treating Nested NER as a latent lexicalized constituency parsing problem, combining structural rigor with the semantic power of head-driven recognition.

Methodology

The core of this framework is the representation of nested entities as lexicalized constituency trees. In this structure, every span is associated with a specific headword, which serves as a vital clue for determining both the boundaries and the type of an entity.

The Eisner-Satta Algorithm

To handle the complexity of lexicalized structures, the authors adapt the Eisner-Satta algorithm for partial marginalization. This allows the model to be trained even when the full "gold" tree structure is unknown and only the entity spans are provided. While the algorithm has a theoretical complexity of $O(n^4)$, the researchers utilize a highly parallelized GPU implementation to ensure the model remains efficient enough for practical use.

Two-Stage Strategy

To address the massive class imbalance—where the vast majority of possible spans in a sentence are not entities—the model employs a two-stage strategy. Instead of predicting structure and labels simultaneously, it first predicts an unlabeled tree to identify potential "things." Only then does it label the identified spans (e.g., as "Person" or "Location"). This focuses the model's computational resources on the most relevant spans.

Head Regularization and Labeling

The model introduces two specific loss functions to refine its understanding of entity heads:

  1. Head Regularization: A soft KL-penalty that encourages the model to follow the general linguistic rule that one head usually governs only one entity.
  2. Head-Aware Labeling: A loss function that calculates the probability of a label based on the distribution of all possible headwords within a given span.

Experimental Results

The model was evaluated on four major Nested NER benchmarks: ACE2004, ACE2005, GENIA (biomedical), and NNE (a dataset with high multi-label frequency).

Performance Highlights

  • State-of-the-Art (SOTA): The model achieved the highest F1 scores to date on ACE2004 (87.90), ACE2005 (86.91), and NNE (94.64).
  • Cross-Domain Robustness: It delivered highly competitive results on the GENIA dataset, proving its effectiveness in specialized domains like biomedical literature.
  • Inference Efficiency: Despite the $O(n^4)$ complexity, the parallelized implementation outperformed sequence-to-set and object-detection-based competitors in speed, and remained comparable to simpler $O(n^3)$ models.

Discussion and Insights

The success of this approach suggests that modeling the dependencies between headwords helps the model predict constituent boundaries more accurately. By using "soft" constraints (like the head regularization loss) rather than hard rules, the model remains robust even when language usage deviates from standard patterns.

While the $O(n^4)$ complexity is managed well through parallelization, it may still present a bottleneck for extremely long documents. Future work could explore extending this "parsing-as-NER" framework to joint entity and relation extraction, where the relationships between headwords are even more critical for understanding the underlying semantics of the text.

Summary updated

Abstract

Nested named entity recognition (NER) has been receiving increasing attention. Recently, (Fu et al, 2021) adapt a span-based constituency parser to tackle nested NER. They treat nested entities as partially-observed constituency trees and propose the masked inside algorithm for partial marginalization. However, their method cannot leverage entity heads, which have been shown useful in entity mention detection and entity typing. In this work, we resort to more expressive structures, lexicalized constituency trees in which constituents are annotated by headwords, to model nested entities. We leverage the Eisner-Satta algorithm to perform partial marginalization and inference efficiently. In addition, we propose to use (1) a two-stage strategy (2) a head regularization loss and (3) a head-aware labeling loss in order to enhance the performance. We make a thorough ablation study to investigate the functionality of each component. Experimentally, our method achieves the state-of-the-art performance on ACE2004, ACE2005 and NNE, and competitive performance on GENIA, and meanwhile has a fast inference speed.
BibTeX
@inproceedings{lou2022nested,
  author = {Chao Lou and Songlin Yang and Kewei Tu},
  title = {{Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=nMdgQrzkmW},
}
Figure