Introduction
Entity Discovery and Linking (EDL) is a critical task in natural language processing that involves identifying entity mentions in text and linking them to a Knowledge Base (KB). Researchers from Zhejiang University (ZJU) participated in the 2016 TAC KBP Trilingual EDL task, focusing on the English portion. Their system, named "WednesdayGO," explores the balance between traditional feature-engineered models and emerging neural architectures, while optimizing the pipeline for speed and accuracy through early-stage coreference resolution.
System Architecture
The WednesdayGO system follows a modular pipeline designed to handle the complexities of discovery and linking sequentially. The process begins with preprocessing and mention extraction, followed by a streamlined linking phase and a final type-inference step to ensure consistency with the Knowledge Base.
Mention Extraction Strategies
The team evaluated two distinct philosophies for the mention extraction phase:
- Traditional CRF (Stanford NER): A linear-chain Conditional Random Field model that utilizes extensive hand-crafted features.
- Neural Network Architecture: An end-to-end model based on the work of Ma and Hovy (2016). This architecture uses a Convolutional Neural Network (CNN) to extract character-level features, which are then concatenated with word embeddings. This combined representation is processed by a Bi-directional LSTM (Bi-LSTM) and a final Chain-CRF layer for tag prediction.
Additionally, the system included a dictionary-based method to identify "nominal mentions"—common noun phrases like "the president"—specifically for person-type entities.
Streamlined Entity Linking
To improve efficiency, the linking module introduces a Simple Coreference Resolution step before candidate generation. By grouping mentions of the same entity within a document, the system only needs to link the most descriptive (usually the longest) mention. This reduces the total number of KB queries and minimizes errors from ambiguous short-form names.
Candidates are generated using Wikipedia redirects, disambiguation pages, and Freebase "also known as" fields. These candidates are then ranked using a linear combination of three metrics:
- Commonness: The prior probability derived from Wikipedia.
- TF-IDF Similarity: Measuring the textual overlap between the document and the entity's description.
- Important Word Hit Rate (IWHR): A custom metric focusing on the overlap of high-value keywords.
Results and Discussion
The evaluation compared two main runs: EDL1 (Stanford NER) and EDL2 (Neural Network).
| Method | NERC (Extraction) F1 | NERLC (Linking) F1 |
|---|---|---|
| EDL1 (Stanford) | 0.750 | 0.682 |
| EDL2 (Neural) | 0.725 | 0.655 |
The "Neural Gap"
Surprisingly, the traditional Stanford NER outperformed the neural model. The authors attribute this to a "domain gap" between the OntoNote 5.0 training data and the specific requirements of the TAC evaluation. This suggests that while neural models are powerful, they require significant domain adaptation to surpass mature, feature-engineered systems in specialized tasks.
The Impact of Nominal Mentions
While the inclusion of nominal mentions (e.g., "the professor") improved extraction scores (NERC), it significantly degraded overall linking performance (NERLC). The highest linking F1 of 0.705 was achieved by the EDL1 configuration that excluded nominal mentions. This indicates that linking common nouns to specific KB entries remains a major bottleneck due to high ambiguity.
Conclusion
The ZJU team's participation highlighted the effectiveness of early coreference resolution in EDL pipelines. While traditional CRF models maintained a slight edge in this competition, the study provides a roadmap for future improvements, particularly in the areas of neural domain adaptation and more sophisticated strategies for nominal entity linking.
Summary updated