Introduction: The Challenge of Global Consistency
Traditional Named Entity Recognition (NER) models typically process text as a linear sequence of words, often at the sentence level. While this "local" approach is effective for many contexts, it inherently ignores the document-level global structure. A significant issue arises when an entity appears multiple times across a document: a model might correctly identify a mention in a clear context but fail in a more ambiguous one. This leads to "global inconsistency," where the same entity is assigned different semantic types within the same document.
For instance, as shown in Figure 1, a sports team name might be correctly identified as an Organization in one sentence but mistakenly labeled as a Location in another. Since coreferent mentions refer to the same real-world entity, they should almost certainly share the same label. Previous architectures lacked a formal mechanism to share information between these distant but related mentions.
Proposed Architecture: Coreference-Aware NER
To address this, the authors propose a novel neural architecture that integrates coreference information directly into the representation learning process. The model builds upon the standard CNN-BiLSTM-CRF pipeline and introduces two key components: a Coreference Layer and Coreference Regularization.
1. Coreference Layer
The Coreference Layer is situated between the BiLSTM and the CRF layer. It takes the hidden states from the BiLSTM and refines them using coreference clusters. For each cluster of mentions, the model calculates a "coreference vector" using max-pooling across all mentions in that cluster. This allows the representation of a specific word—whether it's a repeated name or a pronoun like "he"—to be enriched by the context of every other mention in its group.
2. Coreference Regularization
In addition to the architectural change, the authors introduce a regularization term to the loss function. This term penalizes the model if the learned representations of words within the same coreference cluster are too dissimilar. By encouraging "representational consensus," the model is pushed to learn features that are consistent for the underlying entity throughout the document.
Experimental Setup and Datasets
The model was evaluated on two major NER benchmarks:
- CoNLL-2003: A standard dataset for news-based NER, focusing on Person, Location, Organization, and Miscellaneous types.
- OntoNotes v5.0: A larger, more diverse dataset spanning news, web data, and magazines, featuring 18 fine-grained entity types.
A key challenge is that many NER datasets, like CoNLL-2003, do not include coreference annotations. To test the model's real-world applicability, the authors used the Stanford CoreNLP toolkit to generate "silver" coreference data. This demonstrates that the model can function effectively even without expensive, human-annotated "gold" coreference links.
Key Insights
Robustness to Noise
One of the most compelling findings is the model's resilience to imperfect coreference resolution. Coreference resolution is a difficult task, and automated tools often produce errors. However, the authors found that even noisy coreference links provided enough structural signal to significantly improve NER performance.
Synergy with Contextual Embeddings
The model was also tested alongside ELMo (Embeddings from Language Models). While ELMo provides powerful contextualized word representations, the coreference-aware layers provided additional structural information that ELMo alone did not capture. The combination of the two reached the highest performance levels.
Results and Performance
The coreference-aware model achieved state-of-the-art results on both benchmarks:
- CoNLL-2003: Reached an F1-score of 93.19%.
- OntoNotes v5.0: Reached an F1-score of 89.83%.
Ablation studies confirmed that both the Coreference Layer and the Regularization term contribute significantly to these gains. Furthermore, an error analysis revealed that the model successfully eliminated 20-25% of the errors made by previous top-tier models, specifically those caused by a lack of document-level awareness.
Conclusion
By moving beyond the linear sequence and incorporating document-level coreference information, this research provides a robust solution to the problem of entity inconsistency. The architecture's ability to leverage noisy, automatically generated data makes it a practical choice for improving NER systems in diverse, real-world applications. Future work may explore extending this framework to other relational structures, such as entity-event links, to further enhance deep natural language understanding.
Summary updated