Neural Modeling for Named Entities and Morphology (NEMO^2)

Dan Bareket, Reut Tsarfaty

Introduction

Named Entity Recognition (NER) is a cornerstone of information extraction, typically treating words as atomic units. While this works well for English, it fails significantly in morphologically-rich languages (MRLs) like Hebrew, Arabic, or Turkish. In these languages, entities are often "buried" inside complex "megawords" alongside various prefixes and suffixes. This paper introduces the NEMO^2 framework, a hybrid architecture that demonstrates how morphemes—not tokens—are the essential building blocks for effective NER in MRLs.

The Challenge: Morphological Ambiguity

In Hebrew, a single space-delimited token can contain multiple meaning-bearing units. For example, the token for "and-from-London" is a single string. This presents two primary hurdles:

  1. Boundary Mismatch: Labeling a whole token as a "Location" incorrectly includes grammatical prefixes.
  2. The Circularity Trap: Accurate segmentation often requires knowing if a part of a word is a Named Entity, yet finding the entity requires prior segmentation.
Morphological lattice showing multiple possible segmentations for Hebrew tokens.

As shown in the morphological lattice above, a single Hebrew word can have multiple valid linguistic interpretations. Resolving this ambiguity is critical for downstream tasks like NER.

Modeling Strategies

The authors evaluate three distinct neural strategies using a Bi-LSTM-CRF backbone:

  • Token-single: The standard approach assigning one label per token.
  • Token-multi: Assigning a complex multi-label (e.g., O + B-LOC) to a single token to represent internal morphemes.
  • Morpheme-based: Pre-segmenting text and performing NER directly on individual morphemes.
Comparison of token-level (left) and morpheme-level (right) neural architectures for NER.

The comparison reveals that morpheme-based models are the "Goldilocks" unit, providing the precision necessary to separate entities from their grammatical packaging.

The Hybrid Architecture

The core innovation is a Hybrid Architecture designed to break the circular dependency between segmentation and NER. Instead of a simple linear pipeline, the system uses a token-based NER model to "prune" the morphological search space. By identifying potential entity boundaries first, the model constrains the morphological analyzer to only consider compatible segmentations. A final morpheme-level NER model then provides high-precision labels.

Datasets and Results

To support this research, the authors released a novel parallel NER benchmark for Modern Hebrew consisting of 6,143 sentences from the HAARETZ corpus. This dataset provides annotations at both token and morpheme levels, enabling the first direct empirical comparison of these granularities.

Key results include:

  • State-of-the-Art Performance: The Hybrid model achieved an F1 score of 85.22, a significant improvement over the previous benchmark of 79.10.
  • Improved Segmentation: The NER signal helped improve morphological segmentation accuracy to 90.89%.
  • Generalization: Morpheme-based models effectively solved the "Pseudo-OOV" problem, recognizing known entities even when attached to novel prefixes.

Conclusion

The NEMO^2 framework proves that NER is not just a downstream task but a vital signal for lower-level linguistic analysis. By shifting the focus from tokens to morphemes and utilizing a hybrid pruning strategy, the authors have set a new standard for information extraction in morphologically complex languages. Future work will explore fully joint models to further streamline this process.

Summary updated

BibTeX
@article{bareket2021neural,
  author = {Dan Bareket and Reut Tsarfaty},
  title = {{Neural Modeling for Named Entities and Morphology (NEMO^2)}},
  journal = {Trans. Assoc. Comput. Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00404},
}
Figure