Abstract
Named Entity Typing (NET) is valuable for many natural language processing tasks, such as relation extraction, question answering, knowledge base population, and co-reference resolution. Classical NET targeted a few coarse-grained types, but the task has expanded to sets of hundreds of types in recent years. Existing work in NET assumes that the target types are specified in advance, and that hand-labeled examples of each type are available. In this work, we introduce the task of Open Named Entity Typing (ONET), which is NET when the set of target types is not known in advance. We propose a neural network architecture for ONET, called OTyper, and evaluate its ability to tag entities with types not seen in training. On the benchmark FIGER(GOLD) dataset, OTyper achieves a weighted AUC-ROC score of 0.870 on unseen types, substantially outperforming pattern- and embedding-based baselines.
Introduction
Named Entity Typing (NET) involves assigning semantic labels to entity mentions within a text. While traditional NET systems operate under a "closed-world" assumption—relying on a predefined set of types and extensive labeled data—real-world applications often encounter novel types. This necessitates a shift toward Open Named Entity Typing (ONET), where systems must generalize to unseen labels using only their names and semantic properties.
Existing systems often struggle with zero-shot learning or lack context sensitivity. For instance, a mention of "Chicago" may refer to a city, a musical group, or a film depending on the context. OTyper addresses these challenges by mapping mentions and type names into a shared semantic space, enabling zero-shot classification that accounts for both global patterns and local context.
Methodology
The OTyper architecture is designed to bridge the gap between seen and unseen types by projecting both entity mentions and type labels into a common embedding space.
Mention Representation
The model constructs a comprehensive vector for each entity mention by integrating three components:
- Entity Embedding: The average of the word embeddings of the entity mention.
- Context Embedding: A bidirectional LSTM (Bi-LSTM) equipped with an attention mechanism to capture the specific meaning of the mention based on its surrounding words.
- Hand-crafted Features: A set of 50 features capturing syntactic and topic-based information.
Type Representation and Scoring
Types are represented by averaging the pre-trained word embeddings of their names. This allows the model to leverage the semantic regularity of the embedding space; for example, learning the representation for "painter" provides a useful prior for "sculptor."
The mention vector is projected into the same space as the type vectors. The model is trained to maximize the dot product between a mention and its correct types while minimizing it for incorrect ones. Additionally, OTyper incorporates "Hearst patterns" (e.g., "X such as Y") from web-scale databases to provide external evidence for type relationships.
Experimental Setup
OTyper was evaluated on two major benchmarks: FIGER(GOLD) (113 fine-grained types from Wikipedia) and MSH-WSD (1,387 biomedical types). To rigorously test zero-shot capabilities, the researchers employed 10-fold cross-validation on the types themselves, ensuring that test types were entirely absent from the training phase.
Results and Discussion
OTyper demonstrates significant improvements over traditional methods in zero-shot scenarios. On the FIGER(GOLD) dataset, OTyper achieved a weighted AUC-ROC of 0.870, substantially outperforming both pattern-based and embedding-based baselines.
Ablation studies revealed that while neural context modeling is vital, pattern-based features provide a crucial "world knowledge" component. Removing these features resulted in the most significant performance degradation. Furthermore, the model's performance on unseen types was found to be highly correlated with the semantic similarity between those types and the types encountered during training.
Conclusion
OTyper successfully breaks the "closed-world" barrier of traditional NET by introducing a flexible, context-aware architecture for Open Named Entity Typing. While the model excels at ranking potential types (high AUC-ROC), future work is needed to improve hard-threshold classification (F1 score) in zero-shot settings. Potential improvements include exploring non-linear transformations for type projections to further enhance generalization.
Summary updated