ERICA: Expert Guidance in Validating Crowd Answers
Lists
Only you see this
Abstract
Crowdsourcing became an essential tool for a broad range of Web applications. Yet, the wide-ranging levels of expertise of crowd workers as well as the presence of faulty workers call for quality control of the crowdsourcing result. To this end, many crowdsourcing platforms feature a post-processing phase, in which crowd answers are validated by experts. This approach incurs high costs though, since expert input is a scarce resource. To support the expert in the validation process, we present a tool for \emph{ExpeRt guidance In validating Crowd Answers (ERICA)}. It allows us to guide the expert's work by collecting input on the most problematic cases, thereby achieving a set of high quality answers even if the expert does not validate the complete answer set. The tool also supports the task requester in selecting the most cost-efficient allocation of the budget between the expert and the crowd.
BibTeX
@inproceedings{hung2015erica,
author = {Nguyen Quoc Viet Hung and Duong Chi Thang and Matthias Weidlich and Karl Aberer},
title = {{ERICA: Expert Guidance in Validating Crowd Answers}},
booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
year = {2015},
doi = {10.1145/2766462.2767866},
}Loading related content...