Cost-Saving Effect of Crowdsourcing Learning

Lu Wang, Zhi-Hua Zhou

Abstract

Crowdsourcing is widely adopted in many domains as a popular paradigm to outsource work to individuals. In the machine learning community, crowdsourcing is commonly used as a cost-saving way to collect labels for training data. While a lot of effort has been spent on developing methods for inferring labels from a crowd, few work concentrates on the theoretical foundation of crowdsourcing learning. In this paper, we theoretically study the cost-saving effect of crowdsourcing learning, and present an upper bound for the minimally-sufficient number of crowd labels for effective crowdsourcing learning. Our results provide an understanding about how to allocate crowd labels efficiently, and are verified empirically.
BibTeX
@inproceedings{wang2016cost,
  author = {Lu Wang and Zhi-Hua Zhou},
  title = {{Cost-Saving Effect of Crowdsourcing Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}

Loading related content...

Figure