Abstract
Modern neural architectures for Named Entity Recognition (NER) are highly effective but remain constrained by their reliance on large-scale, human-annotated datasets. This paper, originally presented at EMNLP 2019, proposes GEANN (Gazetteer-Enhanced Attentive Neural Networks), a framework designed to integrate external "name knowledge" from gazetteers to reduce data dependency. By explicitly decoupling entity internal structures from their surrounding contexts, GEANN achieves state-of-the-art performance on nested entity tasks and demonstrates remarkable data efficiency, matching baseline performance with significantly less training data.
Introduction
Named Entity Recognition (NER) is a foundational task in Natural Language Processing, involving the identification and classification of entities such as persons, locations, and organizations. While deep learning has pushed the boundaries of NER, the "data hunger" of these models presents a significant bottleneck. High-quality, human-annotated data is expensive and time-consuming to produce.
Furthermore, traditional sequence labeling models often struggle with nested entities. For instance, in the phrase "George Washington University," "George Washington" is a person nested within an organization. Region-based models address this by evaluating all possible subsequences, but they require even more data to learn the nuances of both entity names and their contexts.
Methodology: Decoupling Knowledge
The core innovation of GEANN is the separation of "name knowledge" (the internal regularity of entity names) from "context knowledge" (how entities are used in sentences). This is achieved through a dual-network architecture.
Attentive Neural Network (ANN)
The ANN component focuses on the relationship between a candidate region and its sentence context. It consists of:
- Inner-Region Encoder: Processes the candidate string itself to capture its internal representation.
- Attentive Context Encoder: Employs an attention mechanism to weigh the importance of surrounding words. This allows the model to determine if the context supports a specific entity type (e.g., the word "said" strongly suggests the preceding entity is a person).
Gazetteer Network
The Gazetteer Network is an auxiliary, context-free component trained on easily obtainable lists of entity names (gazetteers). It learns the compositional regularities of names—for example, identifying that strings ending in "Ltd." are likely organizations—without needing sentential context.
Integration
The final entity representation is a concatenation of the Gazetteer Network's output and the ANN's inner-region representation. This combined feature set is then merged with context features to produce the final classification. This modular design allows the model to leverage "cheap" external knowledge without interfering with its understanding of linguistic structure.
Experimental Evaluation
The authors evaluated GEANN on the ACE2005 dataset, a standard benchmark for nested and overlapping entities. Gazetteers were sourced from Wikipedia anchor texts, providing a large-scale, low-cost knowledge base.
Performance and Data Efficiency
GEANN achieved a state-of-the-art F1-score of 75.2 on ACE2005, outperforming previous region-based, transition-based, and LSTM-CRF models.
A critical finding of the study is the model's data efficiency. As shown in the learning curves, GEANN consistently outperforms baseline models across various training data regimes. Notably, GEANN can achieve performance comparable to fully-trained baseline models while using significantly less annotated data.
Synergy with Pre-trained Models
The study also explored the integration of GEANN with BERT. Even with the powerful contextual representations provided by BERT, GEANN offered a performance boost. This suggests that gazetteers provide specific "context-free" name regularity information that general-purpose language models do not fully capture.
Discussion and Limitations
While GEANN offers significant improvements, it faces some practical constraints. The region-based approach is computationally intensive; to maintain efficiency, the authors limited candidate regions to a maximum length of 6 words. While this covers the vast majority of entities in ACE2005, it may miss longer, complex entities in legal or technical domains. Additionally, the model's effectiveness is tied to the quality and coverage of the available gazetteers.
Conclusion
GEANN demonstrates that decoupling name and context knowledge is a powerful strategy for NER. By effectively incorporating gazetteers into an attentive neural framework, the model reduces the burden of manual annotation and sets a new standard for nested entity recognition. Future work may extend this architecture to cross-lingual or extremely low-resource scenarios where gazetteers are the primary available resource.
Summary updated