Representing a web page as sets of named entities of multiple types

Nan Di, Conglei Yao, Mengcheng Duan, Jonathan J. H. Zhu, Xiaoming Li2008 · DOI

TL;DR

This paper proposes a novel way to represent web pages by focusing on sets of named entities (people, locations, organizations, time) and their relationships, moving beyond the traditional "bag-of-words" approach. This entity-centric model enables more insightful applications like tracking person activity over time and location.

What Problem Does This Paper Address?

Traditional information retrieval and web content mining often represent documents as a "bag of words," which can be too simplistic and lose crucial contextual information. The authors argue that named entities within a web page are more informative and can provide a richer representation. They aim to capture not just the presence of entities but also the relationships between them, enabling deeper analysis and more sophisticated applications that go beyond simple keyword matching.

Methods

The core of the paper is the proposed entity-based page model. This model extracts four types of named entities: person, location, organization, and time. To achieve this, they combine rule-based, model-based (HMM), and Bayesian methods for scalable named entity discovery. For time, they distinguish between page-born time (when the page is available) and page-content time (when the event described happened), extracting the latter from both URLs and page text.

Crucially, they also extract and analyze relations among these entities. They use an unsupervised method based on entity co-occurrence, extending the context by querying search engines with key terms from the co-occurrence context. The extracted relations are then weighted, classified, and labeled. This forms the basis of their entity-based page model, represented as a set of entities and their relations, and a global entity model for a collection of pages.

Datasets

The authors describe a proof-of-concept application using a dataset crawled from the top 100 Chinese news sites (obtained from Alexa.com) over 30 days (October 1, 2007). This resulted in approximately 20 million crawled pages. They extracted entities and relations from this corpus.

Key Insights and Ideas

  • Entities over Keywords: The paper strongly advocates for representing web content through structured entities and their relationships rather than just a collection of words. This allows for a more semantic understanding of the content.
  • Page-born vs. Page-content Time: Distinguishing between when a page was created and when the event it describes occurred is crucial for accurate temporal analysis.
  • Person-Activity Notion: Introducing the concept of "person-activity" (person, location, time, activity) as a structured representation opens up possibilities for tracking individual mobility and behavior.
  • Leveraging Search Engines for Relation Context: Using search engines to expand the context for relation extraction is an innovative way to overcome the limitations of local co-occurrence in large-scale web data.

Key Results

The paper demonstrates two preliminary applications based on their model:

  • Page-Location Mapping: Visualizing the distribution of pages related to specific locations using Google Maps API.
  • Person Friend Searching: Allowing users to explore relationships between prominent individuals, showing the weight and type of their connections. The paper shows an example of tracking the co-occurrence count of political figures over a period, hinting at event detection.

Quantitative results on accuracy or performance metrics are not explicitly detailed in the provided text.

Limitations and Future Work

Limitations: * The paper acknowledges that traditional named entity discovery methods struggle with the complexity and diversity of the web, and their combined approach aims to improve scalability. * The performance of their relation extraction and classification methods on a large-scale web corpus is not quantitatively detailed.

Future Work: * Evolution of Entities and Relations: Investigating how entities and their relationships change over time, as sudden shifts might indicate significant events. * Person Activity Tracking: Further developing the "person-activity" notion to include explicit "action" elements and handle complexities like conflicting information.

Summary updated

Abstract

As opposed to representing a document as a "bag of words" in most information retrieval applications, we propose a model of representing a web page as sets of named entities of multiple types. Specifically, four types of named entities are extracted, namely person, geographic location, organization, and time. Moreover, the relations among these entities are also extracted, weighted, classified and marked by labels. On top of this model, some interesting applications are demonstrated. In particular, we introduce a notion of person-activity, which contains four different elements: person, location, time and activity. With this notion and based on a reasonably large set of web pages, we are able to show how one person's activities can be attributed by time and location, which gives a good idea of the mobility of the person under question.
BibTeX
@misc{di2008representing,
  author = {Nan Di and Conglei Yao and Mengcheng Duan and Jonathan J. H. Zhu and Xiaoming Li},
  title = {{Representing a web page as sets of named entities of multiple types}},
  year = {2008},
  doi = {10.1145/1367497.1367675},
}
Figure