TL;DR
This paper introduces a novel "collective search" approach for disambiguating ambiguous names in text by leveraging the interconnectedness of Wikipedia concepts. By considering all mentions within a document simultaneously, their method achieves more stable and often superior performance compared to existing techniques, particularly in handling concepts not explicitly covered by Wikipedia.
What Problem Does This Paper Address?
In natural language processing, a common challenge is name ambiguity. The same word or phrase can refer to multiple distinct entities or concepts. For instance, "Metropolis" could mean the city from Superman, a movie, or even a physicist. This ambiguity is a significant hurdle for information retrieval systems, knowledge base construction, and semantic search. Existing methods, often called "Wikification," try to resolve this by linking mentions to corresponding Wikipedia articles. However, many of these approaches treat each mention in isolation, failing to exploit the rich contextual information and relationships between multiple mentions within a single document. This paper aims to bridge this gap by proposing a method that considers all mentions collectively to improve disambiguation accuracy and robustness.
Methods
The authors propose a multi-step disambiguation process:
- Collective Search for Candidate Generation: Instead of looking at each mention individually, they create an "ensemble query" using the names of all mentions in a document. This query is used to search a Wikipedia index (built with Lucene). The results are Wikipedia concepts that collectively contain these mentions as link texts. From these, they extract "outlink target concepts" and assign weights based on their occurrence in the collective search results.
- Cross Coherence Calculation: They then compute a "cross coherence" measure to assess how well a candidate concept fits with the other candidate concepts identified for the entire document. This leverages the idea that concepts mentioned together in a document are likely related. They explore different relatedness measures, including normalized Google distance (NGD) and topic-based distances.
- BestFit Candidate Selection: For each mention, they determine a "bestFit" candidate concept by combining the concept's weight (from step 1) and its cross coherence (from step 2).
- Search with Enhanced Queries: The "bestFit" candidate and the mention's own attributes (name, type, context) are used to construct more refined queries for searching Wikipedia. This step aims to retrieve a more accurate set of candidate concepts for each mention.
- Supervised Ranking and Nil-Concept Detection: Finally, a pre-trained Support Vector Machine (SVM) classifier is used to re-rank the candidate concepts. This SVM incorporates various features, including index-based rankings, cross coherence, concept-mention probability, and topic similarity. Crucially, this step also allows for the detection of "nil-concepts" – cases where a mention does not correspond to any Wikipedia article.
Datasets
The authors evaluate their approach on five benchmark corpora:
- MSNBC: A collection of news articles.
- ACE: Another news article dataset.
- AQUAINT: A dataset of news articles.
- CoNLLb: A dataset specifically annotated for named entities.
- IITB: A dataset of news articles.
They note that while these corpora are standard, there can be inconsistencies in annotation schemes and the inclusion of nil-concepts, which they address in their evaluation.
Key Insights and Ideas
- Global Coherence is Key: The core insight is that treating mentions in isolation is suboptimal. By considering the collective context of all mentions in a document, the system can better leverage the interconnectedness of concepts and improve disambiguation accuracy.
- Wikipedia as a Rich Knowledge Source: The paper effectively demonstrates how to harness Wikipedia's structure, including its articles, redirects, and link graph, as a powerful resource for disambiguation.
- Handling Nil-Concepts is Crucial: Many real-world applications encounter mentions that don't have a direct Wikipedia entry. The authors' approach explicitly addresses this by incorporating nil-concept detection, which is often overlooked in prior work.
- Ensemble Queries for Broader Candidate Retrieval: Using an ensemble query that combines all mention names allows for a broader initial retrieval of candidate concepts, increasing the chances of finding the correct one.
- SVM for Fine-Tuning: While the collective search and coherence measures provide a strong foundation, the supervised SVM step is essential for fine-tuning the rankings and accurately identifying nil-concepts, especially when index-based scores alone are insufficient.
Key Results
The paper reports strong performance across all benchmark corpora. For example, on the MSNBC dataset, their best configuration achieved a F1BOT of 89.95%, which is significantly higher than GLOW (74.88%) and M&W (68.49%). Their MAP score was also substantially better. They consistently found that their approach yielded similar or better results compared to other state-of-the-art methods, and importantly, demonstrated the most stable performance across different corpora. The inclusion of context and topic features, along with the bestFit candidate generation, generally improved results, though the impact varied by corpus.
Limitations and Future Work
- Annotation Inconsistencies: The authors acknowledge that benchmark corpora can have annotation inconsistencies (e.g., different links for the same concept, or linking to disambiguation pages), which can affect evaluation and system performance.
- Nil-Concept Handling: While their system handles nil-concepts, they note that some prior work ignores them for evaluation, making direct comparison challenging.
- Future Work: The paper suggests future work could focus on the assignment of concepts not described in Wikipedia, implying a desire to extend their system beyond the current Wikipedia-centric approach.
Summary updated