TL;DR
This paper introduces a game-theoretic model for crowdsourcing contests where distinguishing high-quality submissions is easy, but picking the best one is subjective. They demonstrate that the seemingly minor detail of how ties are broken among high-quality submissions can significantly impact the organizer's utility and the overall efficiency of the contest, challenging the common assumption that tie-breaking rules are inconsequential.
What Problem Does This Paper Address?
Crowdsourcing platforms, like those for design or software development, often present tasks to a large pool of contestants. While it's usually straightforward to filter out low-quality submissions, identifying the single best submission among the good ones can be challenging due to the lack of objective metrics. This paper focuses on a specific aspect of this problem: what happens when multiple contestants submit high-quality work, and the organizer must choose a winner. The prevailing assumption in the literature is that the specific rule used to break ties among these top submissions doesn't matter much, as long as it's consistent. This paper questions that assumption, proposing that the tie-breaking rule can have substantial implications on contestant behavior and the organizer's outcomes.
Methods
The authors develop a complete information, game-theoretic model of crowdsourcing contests. Each contestant has an "ability," modeled as the probability of producing a successful (high-quality) submission. Contestants decide whether to participate based on their expected payoff, considering a fixed participation cost and a reward offered by the organizer. The core of their analysis lies in exploring different deterministic tie-breaking rules. They analyze how these rules influence the set of participants (the Nash equilibrium) for a given reward and how the organizer can choose a reward to maximize its utility. They also investigate the "price of anarchy," which measures the inefficiency of the equilibrium outcome compared to the optimal social welfare.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- Tie-breaking is not trivial: The paper's central argument is that the specific tie-breaking rule used to select a winner from a pool of high-quality submissions is far from a minor detail. It can fundamentally alter the equilibrium participation and the organizer's profits.
- Counterintuitive advantage for weaker players: Perhaps the most surprising finding is that, in many scenarios, an organizer's utility is maximized when ties are broken in favor of less able players. This challenges the intuitive "halo effect" where people assume past success predicts future success.
- "Strongest player first" is a reasonable baseline: While not always optimal, breaking ties in favor of the strongest players (those with the highest ability) provides a guaranteed minimum utility for the organizer, at least 1/3 of the utility achievable with the optimal tie-breaking rule.
- Diversity can hurt efficiency: The paper suggests that when contestants have diverse abilities, the price of anarchy (a measure of inefficiency) can be significantly higher than when contestants have similar abilities. This implies that a diverse talent pool, while potentially offering more options, can lead to greater inefficiencies in contest outcomes.
Key Results
- The paper provides a polynomial-time algorithm for the organizer to compute the optimal reward that maximizes its utility, regardless of the tie-breaking rule.
- They show that for any tie-breaking rule, the organizer's utility is at least 1/3 of the utility achieved with the optimal rule.
- The price of anarchy (PoA) for contests with diverse abilities is bounded by $H_n \approx \ln(n)$, and this bound is tight when ties are broken in favor of stronger players.
- When all players have identical abilities, the PoA is significantly lower, bounded by $3/2$.
Limitations and Future Work
The paper focuses on deterministic tie-breaking rules and complete information about contestant abilities. Future work could explore the implications of random tie-breaking rules, which introduce more complexity as equilibria may no longer be unique. Analyzing these scenarios would require making assumptions about the organizer's expectations regarding which equilibrium will be played. Additionally, extending the model to incorporate incomplete information about contestant abilities would be a valuable next step.
Summary updated