Abstract
Crowdsourcing, in which human intelligence and productivity is dynamically mobilized to tackle tasks too complex for automation alone to handle, has grown to be an important research topic and inspired new businesses (e.g., Uber, Airbnb). Over the years, crowdsourcing has morphed from providing a platform where workers and tasks can be matched up manually into one which leverages data-driven algorithmic management approaches powered by artificial intelligence (AI) to achieve increasingly sophisticated optimization objectives. In this paper, we provide a survey presenting a unique systematic overview on how AI can empower crowdsourcing - which we refer to as AI-Empowered Crowdsourcing (AIEC). We propose a taxonomy which divides algorithmic crowdsourcing into three major areas: 1) task delegation, 2) motivating workers, and 3) quality control, focusing on the major objectives which need to be accomplished. We discuss the limitations and insights, and curate the challenges of doing research in each of these areas to highlight promising future research directions.
Introduction
Traditional crowdsourcing platforms, such as early iterations of Amazon Mechanical Turk, relied heavily on manual matching between requesters (crowdsourcers) and workers. However, as the gig economy scaled through platforms like Uber and Airbnb, manual management became a significant bottleneck. The core challenge lies in a multi-dimensional optimization problem characterized by task complexity, worker heterogeneity, and inherent conflicts of interest between requesters seeking high quality at low cost and workers seeking maximum reward for minimal effort.
As shown in Figure 1, the ecosystem involves a continuous loop of submission, delegation, result submission, quality control, and incentive distribution. To manage these large-scale, long-term ecosystems effectively, "algorithmic management" powered by AI has become essential to optimize the "human-in-the-loop" ecosystem.
A Taxonomy for AI-Empowered Crowdsourcing (AIEC)
This survey proposes a hierarchical taxonomy to categorize AIEC research into three primary functional pillars: Task Delegation, Motivating Workers, and Quality Control. This framework helps organize how AI techniques like reinforcement learning and game theory are making crowdsourcing more efficient and scalable.
1. Task Delegation
AI techniques are used to move from static, offline assignments to dynamic, online models. For Simple Tasks, algorithms often employ Multi-Armed Bandit (MAB) models to iteratively learn worker quality. For Complex Tasks, AI facilitates team formation by identifying workers with complementary skills. Finally, Workflow Optimization manages interdependent sub-tasks to minimize latency and cost.
2. Motivating Workers
Incentive engineering is analyzed through three lenses: Monetary Incentives (e.g., surge pricing, milestone payments), Non-Monetary Incentives (e.g., volunteerism, psychological commitment), and Mixed Incentives (e.g., reputation systems and social prestige). A key insight is that non-monetary factors, such as personal pledges, can often be more effective for long-term engagement than small cash rewards. Furthermore, research on milestone payments shows that the punctuality of rewards is often more important for worker retention than the total amount paid.
3. Quality Control
Modern AIEC shifts from post-hoc aggregation (like simple majority voting) to pre-emptive modeling. This includes Worker Ability Modeling using Bayesian networks or Deep Learning, and Fraud Detection to identify Sybil attacks and collusion by analyzing trajectory patterns in spatial crowdsourcing.
Key Insights and Results
The synthesis of existing literature reveals several critical findings regarding the impact of AIEC:
- Market Scale: The business sector relying on AIEC is projected to reach US$335 billion by 2025.
- Efficiency: Semi-supervised learning for quality assessment can reduce expert labeling effort by up to 60%.
- The "Herding" Effect: Algorithms must actively prevent "task herding," where workers flock to easy tasks. Adaptive acceptance-expectation models help balance the workload across the platform.
- Truthfulness: Game-theoretic mechanisms, such as the Stackelberg Game, have been proven to achieve "truthfulness" and "budget balance" in double auctions for mobile crowdsourcing.
Challenges and Future Directions
Despite significant progress, several gaps remain that define the next frontier of AIEC research:
- Privacy vs. Quality: Techniques like Differential Privacy often degrade data quality. Federated Crowdsourcing is a promising alternative for training models without exposing raw worker data.
- Explainability: Many AIEC algorithms function as "black boxes." For workers to trust the platform, they must understand why they are assigned specific tasks or paid certain amounts.
- Real-world Validation: There is a lack of open-source, real-world testbeds; much current research still relies on simulations or small-scale experiments.
- Ethics and Regulation: The shift to algorithmic management raises urgent questions regarding fair competition and worker rights in the gig economy.
Summary updated