Person resolution in person search results: WebHawk

Xiaojun Wan, Jianfeng Gao, Mu Li, Binggong DingCIKM 2005 · DOI

TL;DR

This paper introduces WebHawk, a system designed to improve person search on the web by resolving the ambiguity of multiple people sharing the same name. It achieves this by filtering out irrelevant pages, clustering remaining pages by individual, and generating informative descriptions for each person, significantly enhancing user experience.

What Problem Does This Paper Address?

The internet is awash with information, and finding a specific person through a general search engine can be a frustrating experience. This is primarily due to two major challenges:

  1. Multi-referent ambiguity: Many people can share the same name (e.g., "David Lee"). Search engines typically return pages for all of them, making it hard to distinguish between individuals.
  2. Junk pages: Search results often include pages that mention a name but aren't actually about a person (e.g., "Ford" referring to the car company, not the person). These "junk pages" clutter results and mislead users.

Existing search engines treat person queries like any other, offering no specialized tools to navigate these ambiguities. This forces users to manually sift through numerous pages, a time-consuming and often fruitless endeavor.

Methods

WebHawk tackles person search through a four-component pipeline:

  1. Filter: This component acts as a first line of defense, identifying and removing "junk pages" that do not contain relevant person information. It uses a Support Vector Machine (SVM) classifier trained on various lexical and linguistic features extracted from the web pages.
  2. Cluster: After filtering, the remaining pages are grouped into clusters, with each cluster ideally representing a single individual. WebHawk employs an agglomerative clustering algorithm that iteratively merges pages based on their similarity. The similarity is calculated using a combination of lexical, linguistic, and extracted personal information features.
  3. Extractor: This component is crucial for enriching the clustering process. It extracts key biographical information from each page, such as the person's full name, title, organization, email address, and phone number. This extraction uses a hybrid approach combining pattern matching with mutual reinforcement learning, particularly for name extraction.
  4. Namer: Finally, the Namer component generates a concise and informative description for each cluster. This description typically includes the person's full name and a key term (often their title or a distinguishing phrase) that helps users quickly identify the individual.

Datasets

The authors used a dataset derived from the logs of MSN search. They selected the 200 most frequent person queries and collected the top 100 web pages retrieved for each query. This dataset was manually annotated for junk pages and referents. For evaluation, they split this data into a training set (160 queries) and a test set (40 queries).

Key Insights and Ideas

  • Junk filtering is critical: The paper strongly emphasizes that removing junk pages is not just a helpful step but a necessary one for effective person search. Without it, clustering performance suffers significantly.
  • Personal information is king: Extracted biographical details (name, title, organization, etc.) are the most powerful features for distinguishing between individuals and improving clustering accuracy.
  • Hybrid approach for extraction: Combining pattern matching with mutual reinforcement learning, especially for name extraction, proves effective in handling the noisy and varied nature of web data.
  • Informative descriptions matter: Simply clustering isn't enough; providing users with clear, concise descriptions of each cluster (the "Namer" component) is vital for usability and helps users quickly find the person they're looking for.

Key Results

  • Junk Filtering: The filter component achieved a precision of 0.803 and a recall of 0.774, with an F1 score of 0.788 when using all features.
  • Personal Information Extraction: WebHawk's extraction methods significantly outperformed a baseline approach, achieving an F1 score of 0.926 for name extraction and 0.906 for organization extraction, compared to 0.698 and 0.574 respectively for the baseline.
  • Clustering: Using a combination of lexical, linguistic, and personal information features on "auto-cleaned data" (data after junk filtering), WebHawk achieved a weighted average F-measure of 0.84, a substantial improvement over using only lexical features (0.76).
  • User Study: In a pilot user study comparing WebHawk with MSN and Vivisimo, WebHawk demonstrated the highest accuracy (82.0%) and the lowest completion time (144.4 seconds) for person search tasks. Users also reported higher satisfaction and preference for WebHawk.

Limitations and Future Work

  • English-centric: Currently, WebHawk only supports English person names. The authors plan to adapt the system for other languages.
  • Clustering Scenarios: They acknowledge that exploring new clustering scenarios could further improve performance.
  • Link Information: Incorporating link information is identified as a potential area for future feature enhancement.
  • System Integration: The authors suggest that general search engines could integrate with WebHawk to offer a dedicated person search option, either through an explicit choice or automatic query identification.

Summary updated

BibTeX
@inproceedings{wan2005person,
  author = {Xiaojun Wan and Jianfeng Gao and Mu Li and Binggong Ding},
  title = {{Person resolution in person search results: WebHawk}},
  booktitle = {ACM International Conference on Information and Knowledge Management},
  year = {2005},
  doi = {10.1145/1099554.1099585},
}
Figure