Integrating web 2.0 resources by wikipedia

Chen Liu, Bing Cui, Anthony K. H. TungACM Multimedia 2010 · DOI

TL;DR

This paper tackles the challenge of integrating diverse Web 2.0 resources (like images and documents) by proposing a novel framework that maps their noisy user-generated tags to high-quality concepts from Wikipedia. This approach aims to create a unified, concept-based representation for seamless exploration across different Web 2.0 platforms.

What Problem Does This Paper Address?

The explosion of Web 2.0 platforms (Flickr, YouTube, Delicious, etc.) has led to an unprecedented amount of user-generated content. While these platforms allow users to share and manage their own resources, they often focus on specific media types. This creates a fragmentation problem: a user interested in a topic might find relevant images on Flickr but miss related documents on Delicious or videos on YouTube. Integrating these disparate resources is crucial for a richer user experience. A common approach is to use user-generated tags, but these tags are notoriously unreliable. They suffer from issues like homonyms (e.g., "apple" for the fruit or the company), synonyms (e.g., "Mac OS X" vs. "Leopard"), and general "noise" from users adding irrelevant tags. Simply linking resources by these noisy tags leads to inaccurate representations and poor integration.

Methods

The paper proposes a two-module framework:

  1. Core Tag Detection: This module aims to identify the most relevant tags for a resource, filtering out the "noisy" ones. It employs a Wikipedia-based topic clustering method. First, it clusters semantically similar tags together, assuming each cluster represents a topic. Then, it ranks these clusters based on their size (number of tags) and coherence (how semantically similar the tags within the cluster are). The tags belonging to the highest-ranked cluster are considered the "core tags."
  2. Context-Based Concept Selection: Once core tags are identified, the system maps them to concepts in Wikipedia. Since a single tag can map to multiple Wikipedia concepts (e.g., "apple" could be the fruit or the company), this module uses a context-based approach to select the most appropriate concept. It leverages the co-occurrence of tags within a resource. For instance, if "apple" appears with "fruit," it's likely referring to the fruit; if it appears with "iPod," it's likely referring to Apple Inc. The system ranks concept candidates based on their correlation with the tag's context and selects the highest-scoring one.

The output is a Uniform Concept-Based Model, where each resource is represented by a set of Wikipedia concepts, abstracting away from the original modality and noisy tags.

Datasets

  • Wikipedia: Used as the knowledge source for concepts. The authors mention using the English version from July 24, 2008, containing approximately 2.7 million concepts.
  • Flickr: Approximately 136,000 images were crawled for evaluation.
  • Delicious: Approximately 114,000 documents were crawled for evaluation.
  • Test Resource Set (TR): A subset of 25 images (IMG) and 25 documents (DOC) from the crawled datasets was created for evaluating the core tag detection algorithm.

Key Insights and Ideas

  • Wikipedia as a Reliable Knowledge Source: The paper highlights Wikipedia's strengths (large coverage, Web 2.0 nature, internal links) as a superior source for semantic concepts compared to noisy user tags.
  • Clustering for Noise Reduction: Using semantic similarity clustering on tags is an effective way to group related terms and identify the central topic of a resource, thereby filtering out irrelevant tags.
  • Context is King for Concept Disambiguation: The core idea that the meaning of a tag is heavily influenced by its surrounding tags is crucial. This context-aware approach significantly improves the accuracy of mapping tags to specific Wikipedia concepts.
  • Uniform Concept Representation: The ultimate goal of representing diverse Web 2.0 resources by a unified set of Wikipedia concepts is a powerful abstraction that enables cross-platform integration and search.

Key Results

  • Tag Coverage: The Wikipedia Inventory (mapping tags to concepts) covers approximately 90% of unique tags from Flickr and Delicious resources, indicating good applicability.
  • Core Tag Detection: The proposed algorithm achieved high precision and recall on the test set, demonstrating its effectiveness in removing irrelevant tags while retaining most of the relevant ones. (Specific numbers are not provided in a summary table, but Figure 1 visually shows high precision and recall for both IMG and DOC).
  • Context-Based Concept Selection: The context-based selection method consistently outperformed a baseline method (ranking concepts by simple co-occurrence frequency) in terms of accuracy for both image (IMG) and document (DOC) resources, as shown in Figure 2.

Limitations and Future Work

  • Limitations: The paper acknowledges that approximately 10% of unique tags are not covered by the Wikipedia Inventory. These often fall into the tail of the tag distribution and can include misspellings, complex phrases, or personalized meanings, which are generally less relevant to a resource's central topic.
  • Future Work: The authors plan to explore more applications of the uniform concept-based model, such as enabling direct resource searching based on this model.

Summary updated

BibTeX
@inproceedings{liu2010integrating,
  author = {Chen Liu and Bing Cui and Anthony K. H. Tung},
  title = {{Integrating web 2.0 resources by wikipedia}},
  booktitle = {ACM Multimedia},
  year = {2010},
  doi = {10.1145/1873951.1874058},
}
Figure