Abstract
Modern Named-Entity Recognition (NER) systems have largely transitioned from manual feature engineering to deep neural networks. While effective, these models often overlook the rich information contained in traditional lexical features or rely on computationally expensive gazetteers. This paper introduces LS (Lexical Similarity) representations, a compact 120-dimensional feature vector that captures the similarity between words and entity types. By embedding words and fine-grained entity types into a joint vector space, the authors achieved a new state-of-the-art F1 score on the OntoNotes 5.0 dataset while maintaining high inference efficiency.
The Limitations of Traditional Gazetteers
Named-Entity Recognition involves identifying and categorizing entities like people, places, and organizations. While neural NER systems are powerful, they often struggle with rare words or lack the domain-specific knowledge found in gazetteers (predefined entity lists). However, standard gazetteers suffer from three major flaws:
- Binary Representation: They provide a simple "yes/no" membership check, ignoring the probability that a word might belong to multiple categories depending on context.
- Computational Overhead: Matching every n-gram in a sentence against massive lexicons is expensive at test time.
- Non-entity Blindness: Gazetteers provide no signal for "trigger words" (e.g., "born", "directed") that are not entities themselves but strongly suggest an entity is nearby.
Methods: LS Representations and Model Architecture
The authors propose a Bi-LSTM-CRF model enhanced by a novel LS representation. The core innovation is a joint vector space where both words and 120 fine-grained entity types (from the WiFiNE resource) reside.
To learn these embeddings, the authors used FastText on a 1.3 billion token corpus. They created two versions of every sentence: the original text and a version where entity mentions were replaced by their type labels. This allows the model to learn proximity between words and the entity types they frequently associate with. For any given word, the LS vector is computed as the cosine similarity between that word’s embedding and each of the 120 entity type embeddings.
Key Insights: Soft Preferences and Contextual Triggers
Unlike a binary gazetteer, the LS vector provides "soft preferences." For example, the word "Hilton" might show high similarity to both /building/hotel and /person/actor, allowing the neural network to resolve the ambiguity using sentence context.
Furthermore, the method captures "contextual triggers." Verbs like "directed" or "won" show high similarity to types like /person/director or /award, providing the model with a strong prior even when the entity itself is rare or unseen.
Results and Performance
The model was evaluated on two major benchmarks:
- OntoNotes 5.0: Achieved a new state-of-the-art F1 score of 87.95, outperforming previous models that used more complex architectures.
- CoNLL-2003: Reached an F1 score of 91.73, matching systems that rely on much heavier language model embeddings.
Because LS features are pre-computed offline, the system maintains high efficiency during inference, avoiding the real-time matching overhead associated with traditional large-scale gazetteers.
Conclusion
LS representations offer a robust, efficient way to re-integrate lexical knowledge into neural NER systems. By moving from binary flags to continuous similarity scores in a joint embedding space, the model gains a nuanced understanding of both entities and the linguistic contexts that signal them. Future work may explore how these static features interact with modern contextualized embeddings like BERT.
Summary updated