Incentives, Motivation, Participation, Games: Human Computation for Linked Data

Katharina Siorpaes, Elena SimperlLDSI@FIA 2010 · OpenReview

Introduction

The "Web of Data" is only as strong as the links between its datasets. While machines have become adept at assigning URIs and publishing metadata, the critical phase of "interlinking"—identifying that different identifiers across databases refer to the same real-world entity—remains a significant bottleneck. This paper argues that the solution lies in Human Computation, specifically through "Games with a Purpose" (GWAP) that transform the tedious work of data interlinking into engaging, competitive experiences.

The Interlinking Bottleneck

Linked Data publishing typically follows three main steps:

  1. Assigning consistent URIs.
  2. Generating links between datasets.
  3. Publishing metadata.

While steps 1 and 3 are largely automatable, step 2—link generation—is notoriously difficult. It requires an understanding of semantic nuances that current algorithms often miss. However, manual interlinking is boring, time-consuming, and expensive. The challenge is to motivate people to contribute their human intelligence to this massive data-alignment task.

Games with a Purpose (GWAP)

To bridge the "automation gap," the authors propose leveraging incentive models by "hiding" the work inside games. This approach, exemplified by the "Incentives Circus" framework, aims to balance three competing interests: gaming fun, the volume of data acquisition, and resulting data quality.

The "Incentives Circus" framework provides a thematic, engaging environment to motivate user participation in data tasks.

Methods and Case Studies

The paper highlights three specific applications of the GWAP paradigm designed to improve the connectivity of the Linked Open Data (LOD) cloud, utilizing sources like DBpedia, Geonames, and Freebase:

  • GuessWhat!: A multiplayer game where players are shown class expressions generated from LOD (e.g., "fruit AND yellow AND grows on tree") and must race to provide a suitable name (e.g., "banana"). This helps build formal vocabularies and ontologies.
  • SpotTheLink: Part of the OntoGame framework, this game tasks players with agreeing on the type of relationship between two concepts, producing mappings between different ontologies.
  • TubeLink: A game designed to bootstrap video annotation. Players choose tags for video content, which the system then uses to select appropriate datasets and instances from the Linked Data cloud.
A gameplay interface where players must associate semantic concepts with visual data, effectively performing the "last mile" of data interlinking.

Key Insights

  • The Automation Gap: A survey of major interlinking tools found that almost all were "semi-automatic," requiring human intervention to achieve high-quality results.
  • The "Last Mile" of Interlinking: Human intuition is uniquely suited for identifying correct link targets and selecting specific "link predicates" (the nature of the relationship).
  • Bootstrapping: Human computation is essential for bootstrapping machine learning methods and validating their results, especially in multimedia contexts where automation is lower.

Conclusion and Future Work

User-contributed interlinking is a viable paradigm for scaling the Web of Data. While building a successful GWAP is a "tricky task" that requires genuine entertainment value to ensure community involvement, games like GuessWhat! and SpotTheLink demonstrate that abstract semantic tasks can be translated into effective competitive mechanics. Future work will focus on refining multimedia interlinking and developing more sophisticated motivation mechanisms to ensure long-term participation.

Summary updated

BibTeX
@inproceedings{siorpaes2010incentives,
  author = {Katharina Siorpaes and Elena Simperl},
  title = {{Incentives, Motivation, Participation, Games: Human Computation for Linked Data}},
  booktitle = {LDSI@FIA},
  year = {2010},
  url = {https://openreview.net/forum?id=2wU6HSBEDo},
}
Figure