Multimedia Crowdsourcing With Bounded Rationality: A Cognitive Hierarchy Perspective

Qi Shao, Man Hon Cheung, Jianwei Huang

This paper explores the intersection of game theory and human psychology in the context of multimedia crowdsourcing. While traditional models assume workers are "fully rational" agents who always make the mathematically optimal choice, this research applies Cognitive Hierarchy (CH) theory to model "bounded rationality"—the reality that humans have limited reasoning depth. The authors demonstrate that platforms can significantly increase their profits by strategically accounting for these human cognitive limits.

TL;DR

By modeling crowdsourcing workers as having limited reasoning levels rather than perfect rationality, this paper shows that requesters can optimize rewards and quality requirements to boost profits. The study reveals that platforms can "exploit" bounded rationality, especially in large populations, where profit continues to grow rather than saturating as it does in traditional models.

What Problem Does This Paper Address?

In multimedia crowdsourcing (e.g., hiring people to take high-resolution photos of specific landmarks), a requester must set two variables: the reward and the quality requirement.

Most existing literature assumes workers are "fully rational," meaning they can perfectly predict how every other worker will behave to reach a Nash Equilibrium (NE). However, real-world experiments show that humans have "cognitive limits." They might only think one or two steps ahead.

The gap this paper fills is twofold:

  1. It integrates quality requirements into the incentive design (most papers only look at rewards).
  2. It replaces the unrealistic "fully rational" assumption with a Bounded Rationality (BR) model to see how it changes the system's performance and the requester's optimal strategy.

Methods

The authors model the system as a two-stage game:

  • Stage I: The requester sets rewards and quality bars for multiple tasks.
  • Stage II: Workers choose which task to perform (or whether to participate at all).

To solve this, they employ two distinct frameworks:

  • Fully Rational (FR) Benchmark: Workers reach a Nash Equilibrium. Because solving this for heterogeneous workers (those with different skill levels) is NP-hard, the authors propose a Greedy Randomized Task Set Search Algorithm to find near-optimal solutions.
  • Bounded Rational (BR) Model: This uses Cognitive Hierarchy (CH) Theory. Workers are categorized into levels ($k=0, 1, 2, \dots$).
  • Level-0 workers choose tasks randomly.
  • Level-1 workers assume everyone else is Level-0 and choose the best response.
  • Level-$k$ workers assume the population follows a Poisson distribution of lower-level thinkers.
  • Convergence Analysis: The authors mathematically prove that the FR model is actually just a "limiting case" of the BR model—as the average cognitive level ($\tau$) of the population goes to infinity, the BR equilibrium converges to the Nash Equilibrium.

Datasets

The paper relies on numerical simulations rather than real-world datasets. The authors simulate scenarios with varying numbers of tasks ($M=3, 5, 10, 20$) and worker populations ($N$ up to 300) to validate their algorithms and theoretical observations.

Key Insights and Ideas

  • The Profit of Imperfection: Requesters can actually make more profit when workers are less rational. In a fully rational world, workers perfectly coordinate to maximize their own payoffs. In a boundedly rational world, workers often "over-compete" for tasks, allowing the requester to lower rewards while still maintaining high participation.
  • The Saturation Gap: In traditional (FR) models, the requester's profit eventually saturates as the worker population grows. However, in the BR model, profit can increase unboundedly with the population size. This is because there is always a fresh supply of "Level-0" thinkers who will participate for minimal rewards.
  • Strategic Reward Reduction: There is a "threshold" population size where a requester can drastically cut rewards. If the population is large enough, the requester can rely almost entirely on low-level thinkers who don't strategically avoid low-reward tasks.
  • Quality Differentiation: The authors found that unless one type of worker (high-skill or low-skill) completely dominates the platform, the requester should always set different quality requirements for different tasks to maximize revenue.

Key Results

  • Heuristic Efficiency: The proposed Greedy Randomized algorithm for the FR model achieved results nearly identical to an exhaustive search but with much lower computational complexity.
  • Exploitation Potential: Simulations showed that when the average cognitive level ($\tau$) is low (e.g., $\tau=1$), the requester can achieve significantly higher profits compared to the Nash Equilibrium benchmark.
  • Threshold Effect: In one simulation ($\tau=2$), once the worker population $N$ hit 200, the requester could reduce the reward to the bare minimum (the worker's cost $c_1$) and still get the task done, whereas in the FR model, the reward had to remain much higher to attract rational agents.

Limitations and Future Work

  • Information Symmetry: The model assumes the requester has complete information about the workers' costs and quality capabilities, which is rarely true in practice.
  • Static Costs: The costs are task-dependent but not worker-dependent. In reality, a professional photographer has different costs than an amateur.
  • Future Directions: The authors suggest extending the model to include rewards that depend on the actual effort exerted and conducting field trials to estimate the real-world average cognitive level ($\tau$) for different types of crowdsourcing platforms.

Summary updated

Abstract

In multimedia crowdsourcing, the requester's quality requirements and reward decisions will affect the workers' task selection strategies and the quality of their multimedia contributions. In this paper, we present a first study on how the workers' bounded cognitive rationality interacts with and affects the decisions and performance of a multimedia crowdsourcing system. Specifically, we consider a two-stage model, where a requester first determines the reward and the quality requirement for each task, and the workers select the tasks to accomplish accordingly. First, we consider the benchmark case where users are fully rational, and derive the requester's optimal rewards and quality requirements for the tasks. Furthermore, we focus on the more practical bounded rational case by modeling the workers' task selection behaviors using the cognitive hierarchy theory. Comparing with the fully rational benchmark, we show that the requester can increase her profit by taking advantage of the workers' bounded cognitive rationality, especially when the workers' population is large or the workers' average cognitive level is low. When the workers' average cognitive level is very high, however, the equilibrium under the practical bounded rational model converges to that under the benchmark fully rational model. It is because the workers at different levels make decisions sequentially and high cognitive level workers can accurately predict other users' strategies. Under both the fully and bounded rational models, we show that if workers are heterogeneous but one type of workers (either the high or the low quality) dominates the platform, the requester cannot make a higher profit by setting different quality requirements for different tasks.
BibTeX
@article{shao2019multimedia,
  author = {Qi Shao and Man Hon Cheung and Jianwei Huang},
  title = {{Multimedia Crowdsourcing With Bounded Rationality: A Cognitive Hierarchy Perspective}},
  journal = {IEEE J. Sel. Areas Commun.},
  year = {2019},
  url = {https://openreview.net/forum?id=7Gc7umSbMv},
}
Figure