Abstract
Taylor Cassidy, Heng Ji, Lev-Arie Ratinov, Arkaitz Zubiaga, Hongzhao Huang. Proceedings of COLING 2012.
2012.
Introduction
"Wikification," formally known as Disambiguation to Wikipedia (D2W), is the process of automatically linking text mentions to their corresponding Wikipedia entries. While this task is well-established for long-form news articles, it is notoriously difficult for microblogs like Twitter. The inherent brevity and informal nature of tweets often strip away the linguistic clues that algorithms—and even humans—rely on for accurate linking. This paper investigates how "Context Expansion" can overcome these limitations by leveraging information beyond the bounds of a single 140-character message.
The Challenge of Context Sparsity
Traditional D2W systems rely on surrounding text to resolve ambiguity. For example, a news article mentioning "Hawks" would likely contain other keywords like "NBA," "Atlanta," or "playoffs," allowing a system to distinguish the basketball team from the bird or the NHL's Chicago Blackhawks.
In contrast, a tweet might simply read, "Go Hawks!!" This "context sparsity" makes it nearly impossible to identify the correct entity without additional data. As illustrated in the following table, mentions often appear with varying levels of internal context, ranging from completely isolated mentions to those supported by the author's broader tweet history.
Methods: Strategies for Context Expansion
To provide disambiguation algorithms with more data, the researchers utilized a state-of-the-art system called GLOW (Global and Local Object Wikification). GLOW attempts to maximize "global coherence," ensuring that all entities identified within a document are logically related to one another. The authors tested three distinct methods for defining the "document" context fed into the system:
- By File (Baseline): Each tweet is disambiguated in isolation. This represents the standard approach and serves as the control group.
- By Author: All tweets from a single user are grouped into a single document. This method assumes that a user's interests (e.g., sports, politics, or music) remain consistent over time, providing a stable thematic background.
- By Cluster: Tweets are grouped by topic using a hybrid approach involving Probabilistic Latent Semantic Analysis (PLSA) and Random Walk-based Propagation (PRP). This helps smooth out topic distributions to overcome the brevity of individual tweets.
Datasets and Refinement
The study utilized two primary benchmarks:
- Gold1: A dataset of 502 tweets from 28 verified accounts, containing 812 manual annotations, originally created by Meij et al. (2012).
- Gold2: A refined version created by the authors. They discovered that human annotators often missed valid links in the noisy Twitter environment. By manually reviewing system "false positives," they identified and added 70 missing correct links to create a more robust benchmark.
Key Results and Insights
The researchers measured performance using the F1-measure (the harmonic mean of precision and recall).
- Baseline Performance: The "By File" approach achieved an F1 of 45.58% on the Gold1 dataset.
- The Superiority of Author Context: The most significant finding was that grouping by author outperformed grouping by topic. The "By Author" expansion reached an F1 of 48.07% on Gold1 and 53.92% on the more complete Gold2 dataset.
- Topical Clustering: The "By Cluster" method also improved upon the baseline, achieving an F1 of 47.50%.
These results highlight two critical points. First, a user's past tweets are a highly reliable signal for what they mean in their current tweet. Second, even "gold standard" human-annotated datasets for Twitter are often incomplete; humans frequently overlook mentions of songs, dates, or acronyms in the chaotic microblogging environment.
Conclusion and Future Work
While microblog Wikification remains a difficult task compared to news-based D2W (which often sees F1 scores in the 80-90% range), context expansion represents a major step forward. The researchers demonstrated that by looking at the author's history or topical clusters, we can bridge the context gap. Future work will focus on better exploiting Twitter-specific features like hashtags and URLs, as well as refining how systems ignore "chatter" that has no relevant Wikipedia equivalent.
Summary updated