Abstract
This paper presents a large-scale system for the recognition and semantic disambiguation of named entities based on information extracted from a large encyclopedic collection and Web search results. It describes in detail the disambiguation paradigm employed and the information extraction process from Wikipedia. Through a process of maximizing the agreement between the contextual information extracted from Wikipedia and the context of a document, as well as the agreement among the category tags associated with the candidate entities, the implemented system shows high disambiguation accuracy on both news stories and Wikipedia articles.
Introduction
Identifying a name in a text is only the first step in information extraction. While a Named Entity Recognition (NER) system might correctly identify "Texas" as a location, it does not inherently resolve which "Texas" is being referenced—be it the U.S. state, the University of Texas at Austin, or even a pop band. This challenge, known as Named Entity Disambiguation (NED), becomes increasingly difficult as systems scale to the vast, diverse content of the web.
This research addresses semantic ambiguity by mapping mentions to a world-scale reference list: Wikipedia. By leveraging Wikipedia's structured and unstructured data, the authors developed a system capable of resolving mentions against a pool of over 1.4 million entities.
System Architecture
The system operates through a modular pipeline designed to move from raw text to disambiguated entities.
As shown in the architecture above, the process involves three main stages:
- Truecasing and Sentence Breaking: Preparing the text using web statistics and gazetteers.
- Entity Recognition: A hybrid NER component identifies mention boundaries and assigns broad labels (e.g., Person, Location) using CoNLL data and web search query logs.
- Entity Disambiguation: The final stage where candidate entities are ranked and selected based on Wikipedia context and tag information.
Knowledge Extraction and Representation
The core of the system is its massive knowledge base extracted from the English Wikipedia. The system stores three primary types of clues for each entity:
- Surface Forms: The various ways an entity might be mentioned in text.
- Category Tags: Extracted from Wikipedia’s category hierarchy.
- Contextual Clues: Words and other entities that frequently co-occur with the entity in Wikipedia articles.
Figure 1 illustrates how an ambiguous surface form like "Columbia" can map to multiple distinct entities. The system uses the associated tags (e.g., "Manned spacecraft") and contexts (e.g., "NASA") to distinguish between "Space Shuttle Columbia," "Columbia University," or "Columbia River."
Disambiguation via Agreement Maximization
The disambiguation engine employs a Vector Space Model to represent the document and candidate entities. Rather than resolving mentions in isolation, the system seeks to maximize "agreement" across the entire document.
If a document mentions "Columbia" and "Discovery," the system identifies them as space shuttles because they share astronomical and NASA-related category tags. This "category agreement" is one of the most powerful signals for disambiguation. Furthermore, the system handles local context shifts by iteratively "shrinking" its focus from the document level to the paragraph or sentence level if a global consensus is not found.
Evaluation and Results
The system was evaluated using both Wikipedia articles and a "News Evaluation Set" consisting of top stories from MSNBC.
- News Accuracy: The system achieved 91.4% accuracy on news data, significantly outperforming a 51.7% baseline. This demonstrates the necessity of sophisticated context for resolving common, highly ambiguous names in news reporting.
- Wikipedia Accuracy: On Wikipedia articles, the system reached 88.3% accuracy. While the improvement over the baseline (86.2%) seems smaller here, the system showed marked improvement on specifically ambiguous surface forms.
- Scale: At the time of publication, resolving mentions against 1.4 million entities was a significant engineering feat.
Conclusion
By treating Wikipedia as a structured knowledge graph rather than just a text corpus, the authors demonstrated that world-scale NED is achievable. The system's reliance on category agreement and contextual vectors provides a robust framework for resolving ambiguity. While limited by Wikipedia's coverage and occasional category noise, the approach highlights the importance of dynamic, large-scale knowledge bases in modern NLP.
Summary updated