Towards Robust Unsupervised Personal Name Disambiguation

Ying Chen, James MartinEMNLP 2007

TL;DR

This paper tackles the challenge of distinguishing between different people who share the same name in large collections of online documents. They introduce a novel approach that leverages a combination of syntactic and semantic features, going beyond simple keyword matching to achieve more robust disambiguation for both English and Chinese texts.

What Problem Does This Paper Address?

In today's digital age, we're drowning in information from vast, open-domain sources like the web. Many applications, such as question answering, summarization, and information extraction, rely on these documents. A common hurdle is dealing with ambiguity, especially when queries involve names of people, places, or organizations. For instance, a search for "John Smith" might return documents about multiple distinct individuals named John Smith. Without a way to separate these mentions, downstream applications will produce inaccurate results. This paper focuses specifically on the problem of personal name disambiguation in news articles, aiming to automatically group documents that refer to the same individual.

Methods

The authors propose an unsupervised clustering approach that combines syntactic and semantic processing. Their methodology follows a standard pipeline:

  1. Ambiguous Object Detection: They assume the ambiguous personal name has already been identified. Following a "one person per document" policy, an "ambiguous object" is defined as a set of target entities within a document that share the ambiguous name.
  2. Feature Extraction and Representation: This is the core of their contribution. They extract features along two dimensions:
  3. Syntax to Semantics: They go beyond simple "bag-of-words" by using syntactic phrase chunking and a named-entity recognition (NER) and co-reference resolution system (EXERT). This allows them to capture more nuanced information like noun phrases that represent a person's job or location.
  4. Local to Document-Level Contexts: They extract features from both local sentences (surrounding the name mention) and summarized sentences (headline + first sentence) to capture immediate context, as well as broader document-level information. They define several feature types:

  5. Base Noun Phrase Features: These are noun phrases found in local or summarized sentences.

  6. Named-Entity Features: These include "target entities" (the entity with the ambiguous name), "local entities" (other entities closely related to the target entity), and "non-local entities" (entities providing background information).
  7. Similarity Matrix Learning: They use a hybrid similarity scheme called SoftTFIDF, which combines TF-IDF weighting with the Jaro-Winkler distance. For English, which uses phrase-based features, they employ a "two-level SoftTFIDF."
  8. Clustering: They use agglomerative clustering with a single linkage method.

What's novel here is the sophisticated feature engineering that combines syntactic parsing and robust named-entity recognition to extract richer semantic information than previous methods, and the application of these techniques to both English and Chinese.

Datasets

The paper uses two primary datasets:

  • Bagga's Corpus: A collection of 256 articles from the New York Times, originally used by Bagga & Baldwin (1998) for the English personal name "John Smith."
  • Boulder Name Corpus: A newly collected news corpus from web searches, designed to be more heterogeneous and noisy than Bagga's corpus. It includes four English personal names (James Jones, John Smith, Michael Johnson, Robert Smith) and four Chinese personal names (Li Gang, Li Hai, Liu Bo, Zhang Yong), with 100 news articles retrieved for each.

The Boulder Name Corpus is presented as a more realistic benchmark due to its diverse sources.

Key Insights and Ideas

  • Rich features are crucial: The paper strongly suggests that moving beyond simple token-based features to incorporate syntactic and semantic information significantly improves disambiguation performance.
  • Document-level context matters: Features derived from the entire document (like document entities) are more effective than relying solely on local sentence context.
  • Language differences exist: While the proposed methods work for both English and Chinese, there are interesting differences in how features contribute, likely due to linguistic variations (e.g., pronoun usage, word segmentation).
  • Long-tailed distributions are challenging: The prevalence of singleton clusters (entities mentioned only once) in real-world data poses a challenge for standard clustering algorithms.

Key Results

The authors report significant improvements in disambiguation performance across various feature combinations.

  • On Bagga's corpus, their best model ("Bagga + summarized bnp + document entities") achieved a B-cubed F score of 92.02%, outperforming the reported best result of Bagga & Baldwin (1998) by 7.42%.
  • On the Boulder Name Corpus, their best model ("Bagga + summarized bnp + document entities") achieved average B-cubed F scores of 93.77% for English and 97.41% for Chinese.
  • They found that incorporating syntactic features (contextual bnp, summarized bnp) and semantic features (contextual entities, document entities) consistently improved performance.

Limitations and Future Work

  • Clustering Algorithm: The authors acknowledge that their current clustering approach (agglomerative clustering) might not be optimal for the skewed, long-tailed distribution of their data. They suggest that clustering algorithms specifically tailored to such distributions are needed.
  • Feature Engineering: While their features are robust, they believe there's room to explore additional semantic features, such as those derived from semantic relations and semantic role labels.
  • Data Scope: The current work is focused on news articles. Future work could extend these techniques to other types of web pages that might contain even noisier information.
  • Parameter Setting: While they explored optimal stop-thresholds, they note that learning optimal weighting schemes for combining different similarity matrices for each personal name is difficult and was not pursued in this paper.

Summary updated

BibTeX
@inproceedings{chen2007towards,
  author = {Ying Chen and James Martin},
  title = {{Towards Robust Unsupervised Personal Name Disambiguation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2007},
}
Figure