Crowdsourcing Feature Discovery via Adaptively Chosen Comparisons
Lists
Only you see this
Abstract
We introduce an unsupervised approach to efficiently discover the underlying features in a data set via crowdsourcing. Our queries ask crowd members to articulate a feature common to two out of three displayed examples. In addition, we ask the crowd to provide binary labels for these discovered features on the remaining examples. The triples are chosen adaptively based on the labels of the previously discovered features on the data set. This approach is motivated by a formal framework of feature elicitation that we introduce and analyze in this paper. In two natural models of features, hierarchical and independent, we show that a simple adaptive algorithm recovers all features with less labor than any nonadaptive algorithm. The savings are as a result of automatically avoiding the elicitation of redundant features or synonyms. Experimental results validate the theoretical findings and the usefulness of this approach.
BibTeX
@inproceedings{zou2015crowdsourcing,
author = {James Y. Zou and Kamalika Chaudhuri and Adam Tauman Kalai},
title = {{Crowdsourcing Feature Discovery via Adaptively Chosen Comparisons}},
booktitle = {HCOMP},
year = {2015},
doi = {10.1609/hcomp.v3i1.13231},
}| AdaFlock: Adaptive Feature Discovery for Human-in-the-loop Predictive Modeling | AAAI · 2018 |
| Crowdclustering | NeurIPS · 2011 |
| Adaptively Learning the Crowd Kernel | arXiv · 2011 |
| Crowdsourced Clustering via Active Querying: Practical Algorithm with Theoretical Guarantees | AAAI · 2023 |
| Optimal clustering from noisy binary feedback | Machine Learning · 2024 |
| Learning Attributes from the Crowdsourced Relative Labels | AAAI · 2017 |