TL;DR
This paper provides a theoretical framework for efficiently managing crowdsourced innovation when both task difficulty and worker skill are unknown. The authors prove that the optimal strategy is a "bottom-up" hierarchy where easy-to-access, lower-skilled workers attempt tasks first, and higher-skilled experts only step in after others have failed—a process naturally incentivized by a simple mechanism where task rewards increase every time a task remains uncompleted.
What Problem Does This Paper Address?
Crowdsourcing platforms like Kaggle or Amazon Mechanical Turk face a fundamental resource allocation dilemma. High-skill workers are a scarce and expensive resource. If a world-class data scientist spends their time on a task that a college student could have solved, that expert's time is wasted.
In a traditional firm, a manager might assign tasks based on known resumes. However, in "crowd innovation," two major hurdles exist:
- Information Asymmetry: The employer doesn't know the specific skills of the anonymous workers in the crowd.
- Task Uncertainty: The difficulty of an innovation task (like folding a protein or finding a gold deposit) often isn't known until someone actually tries to solve it.
The paper asks: How can we design a marketplace that automatically sorts the right workers to the right tasks without a central manager knowing everyone's "stats"?
Methods
The authors develop a mathematical model of a virtual marketplace using a continuum limit (treating the large pool of workers and tasks as continuous measures). They analyze the problem in two stages:
- The Assignment Problem: If a social planner knew everyone’s skills and every task’s difficulty, what would the optimal matching look like? They use dynamic programming and a novel "Swapping Lemma" to prove that a hierarchical structure is optimal.
- The Implementation Problem: How do you achieve that optimal matching in a decentralized market where workers choose their own tasks? They propose a dynamic pricing mechanism where the reward for a task ($p$) increases each time it is attempted and not finished.
The core mathematical engine relies on a comparative advantage property: more skilled workers are relatively better at difficult tasks than less skilled workers. This allows the authors to define a "Group Equilibrium" where workers self-select into the appropriate layer of the hierarchy based on the expected payoff.
Datasets
Not discussed in the paper. This is a theoretical microeconomic paper. The authors use stylized numerical examples (e.g., "Example 1" with uniform distributions of skill and difficulty) to illustrate their proofs, but they do not use empirical datasets or training sets.
Key Insights and Ideas
- The "Filtering" Effect: The optimal hierarchy is "bottom-up." By letting low-skill workers try tasks first, they "filter out" the easy problems. This ensures that when a high-skill worker finally engages, they are only working on tasks that actually require their expertise.
- Prices as a Sorting Signal: You don't need to interview workers. By increasing the prize money for a task every time it fails, you naturally push low-skill workers to try things early (when the prize is small but the task might be easy) and induce high-skill workers to wait for the "hard" tasks that have accumulated a larger bounty.
- The Anti-Pyramid: Unlike traditional corporate hierarchies that are usually pyramidal (many people at the bottom, few at the top), a crowd hierarchy can be any shape. Depending on the distribution of skills, it might be optimal to have a very small first layer and a massive middle layer.
- The "More is Less" Paradox: In many organizational models, adding more layers of management or review is always better. Here, adding too many layers can actually decrease total innovation. Why? Because every extra layer exposes more workers to "impossible" tasks, wasting their time and preventing them from working on tasks they could actually solve.
Key Results
- Theorem 1 (Optimality of Hierarchy): The authors prove that for any set of tasks and workers, there exists a hierarchical matching that maximizes total output.
- Theorem 2 (Decentralized Implementation): They prove that a simple pricing scheme—charging an entry fee ($q$) and offering tiered payments ($p_1 < p_2 < ... < p_k$) for tasks that survive successive rounds—can perfectly replicate the "first-best" optimal assignment.
- Stochastic Dominance: They demonstrate that as tasks move through the hierarchy, the "difficulty distribution" of the remaining tasks shifts toward being harder in a first-order stochastic dominance sense. This confirms that the "market" is effectively learning about task difficulty through the failures of its participants.
Limitations and Future Work
- Effort and Moral Hazard: The model assumes a worker's success is purely a function of skill vs. difficulty. It doesn't account for "shirking" or workers who might intentionally fail to drive up the price (though the authors argue that competition in a large crowd mitigates this).
- Reputation: The current model is anonymous. The authors note that adding a reputation system (like those on Upwork) would change the dynamics, potentially allowing for more efficient "top-down" assignments for known experts.
- Unstructured Tasks: The results rely on "structured" tasks where skill is cumulative (if you can solve a hard task, you can solve an easy one). The authors admit that for "unstructured" tasks—where success is more about luck or a "niche" perspective—the hierarchical model might break down.
- Empirical Validation: As a theoretical work, the next logical step is for researchers to test these pricing mechanisms in real-world crowdsourcing environments to see if workers actually respond to the "price-up-on-failure" signal as predicted.
Summary updated