Harvesting Entities from the Web Using Unique Identifiers - IBEX

Aliaksandr Talaika, Joanna Biega, Antoine Amarilli, Fabian M. Suchanek

TL;DR

This paper introduces IBEX, a novel system for systematically harvesting unique entity identifiers (like ISBNs, GTINs, DOIs) and their corresponding human-readable names from the web at scale. IBEX achieves high accuracy (73-96%) and broad coverage, creating a valuable database for analyzing entity distribution and enabling new forms of web-scale knowledge discovery.

What Problem Does This Paper Address?

The web is a vast repository of information about entities – products, people, books, and more. However, extracting and organizing this information in a structured, reliable way is challenging. While unique identifiers (like ISBNs for books or GTINs for products) exist, associating them with their correct, human-readable names on web pages is difficult. Web pages often contain multiple names, some of which might be incorrect or ambiguous. Existing methods struggle with the scale and inherent messiness of the web, making it hard to build comprehensive, accurate knowledge bases of entities. This paper tackles the problem of reliably extracting these entity-name pairs at web scale, going beyond existing domain-specific approaches.

Methods

IBEX employs a three-phase approach:

  • Phase 1: Extraction of IDs and Name Candidates: This phase uses Named Entity Recognition (NER) to identify potential IDs and candidate names on web pages. It then scores these candidates based on various features (e.g., proximity to the ID, tag information) to create an initial, noisy list of potential entity-name pairs.
  • Phase 2: Filtering Erroneous Names: This phase leverages the core insight that correct entity names tend to be strongly associated with a specific ID, while incorrect or generic names appear more uniformly across many IDs. It identifies and removes names that are too broadly distributed, significantly reducing noise.
  • Phase 3: Selecting the Best Name: For each ID, this final phase selects the most frequent candidate name. If there's a tie in frequency, the name with the highest score from Phase 1 is chosen.

This multi-stage filtering process is designed to be robust, scalable, and applicable to diverse entity types without requiring domain-specific knowledge or training data for each new type.

Datasets

The primary dataset used is the ClueWeb corpus, specifically the English portions of ClueWeb09 and ClueWeb12, totaling approximately 35 TB and 1.2 billion web pages. For evaluation, a gold standard dataset was created by randomly sampling 200 occurrences of IDs for each entity type (GTIN, CAS, DOI, Email) and manually annotating them with their correct names.

Key Insights and Ideas

  • The Power of Unique Identifiers: The paper demonstrates that unique identifiers, despite their varied formats, are powerful anchors for extracting structured information from the web. Their inherent uniqueness is key to disambiguating entities.
  • Name Specificity as a Signal: The core idea of Phase 2 is that correct entity names are specific to an ID, while noise (generic terms, descriptive words) is general. This statistical property is crucial for filtering.
  • Scalability Through Phased Filtering: The three-phase approach allows for initial broad extraction followed by increasingly refined filtering, making the system scalable to billions of web pages while maintaining high accuracy.
  • Domain Agnosticism: The method is designed to work across various entity types (products, chemicals, documents, people) without needing type-specific rules or training data, making it highly generalizable.

Key Results

  • High Accuracy and Coverage: IBEX achieves impressive accuracy rates, ranging from 73% for documents to 96% for chemical substances. The system successfully extracts millions of unique entities.
  • Database Creation: The output is a database containing millions of uniquely identified entities, with associated names and URLs of pages where they were found. This database is significantly larger than existing public knowledge bases.
  • Quantitative Performance:
    • GTINs: Accuracy increased from 38% in Phase 1 to 82% in Phase 3.
  • CAS: Accuracy increased from 76% in Phase 1 to 96% in Phase 3.
  • DOIs: Accuracy increased from 52% in Phase 1 to 73% in Phase 3.
  • Emails (pseudo-ids): Accuracy remained high at 90% throughout the phases.
  • Novel Analyses: The created database enables new analyses, such as mapping product production by country, analyzing global trade flows, and studying common person names and gender distribution on the web.

Limitations and Future Work

  • Pseudo-IDs: For entity types like email addresses, where one entity can have multiple identifiers (e.g., a person with several email accounts), the system uses "pseudo-IDs." This means it cannot guarantee the uniqueness of entities in the database for these types.
  • NER Module Dependency: The performance of IBEX is dependent on the quality of the underlying Named Entity Recognition (NER) modules used for identifying IDs and candidate names.
  • Future Work: The authors suggest extending the approach to extract entity attributes (e.g., product prices, chemical formulas) and applying the methodology to a broader range of entity types (banks, companies, audiovisual material, social network IDs) and corpora. They also highlight the potential for making the web more semantic and useful.

Summary updated

Abstract

In this paper we study the prevalence of unique entity identifiers on the Web. These are, e.g., ISBNs (for books), GTINs (for commercial products), DOIs (for documents), email addresses, and others. We show how these identifiers can be harvested systematically from Web pages, and how they can be associated with human-readable names for the entities at large scale. Starting with a simple extraction of identifiers and names from Web pages, we show how we can use the properties of unique identifiers to filter out noise and clean up the extraction result on the entire corpus. The end result is a database of millions of uniquely identified entities of different types, with an accuracy of 73--96% and a very high coverage compared to existing knowledge bases. We use this database to compute novel statistics on the presence of products, people, and other entities on the Web.
BibTeX
@inproceedings{talaika2015harvesting,
  author = {Aliaksandr Talaika and Joanna Biega and Antoine Amarilli and Fabian M. Suchanek},
  title = {{Harvesting Entities from the Web Using Unique Identifiers - IBEX}},
  booktitle = {Webdb},
  year = {2015},
  doi = {10.1145/2767109.2767116},
}
Figure