Statistical Quality Control for Human Computation and Crowdsourcing

Yukino Baba

Abstract

Human computation is a method for solving difficult problems by combining humans and computers. Quality control is a critical issue in human computation because it relies on a large number of participants (i.e., crowds) and there is an uncertainty about their reliability. A solution for this issue is to leverage the power of the "wisdom of crowds"; for example, we can aggregate the outputs of multiple participants or ask a participant to check the output of another participant to improve its quality. In this paper, we review several statistical approaches for controlling the quality of outputs from crowds.
BibTeX
@inproceedings{baba2018statistical,
  author = {Yukino Baba},
  title = {{Statistical Quality Control for Human Computation and Crowdsourcing}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/806},
}

Loading related content...

Figure