Becoming the Super Turker: Increasing Wages via a Strategy from High Earning Workers

Saiph Savage, Chun-Wei Chiang, Susumu Saito, Carlos Toxtli, Jeffrey P. Bigham

Abstract

Crowd markets have traditionally limited workers by not providing transparency information concerning which tasks pay fairly or which requesters are unreliable. Researchers believe that a key reason why crowd workers earn low wages is due to this lack of transparency. As a result, tools have been developed to provide more transparency within crowd markets to help workers. However, while most workers use these tools, they still earn less than minimum wage. We argue that the missing element is guidance on how to use transparency information. In this paper, we explore how novice workers can improve their earnings by following the transparency criteria of Super Turkers, i.e., crowd workers who earn higher salaries on Amazon Mechanical Turk (MTurk). We believe that Super Turkers have developed effective processes for using transparency information. Therefore, by having novices follow a Super Turker criteria (one that is simple and popular among Super Turkers), we can help novices increase their wages. For this purpose, we: (i) conducted a survey and data analysis to computationally identify a simple yet common criteria that Super Turkers use for handling transparency tools; (ii) deployed a two-week field experiment with novices who followed this Super Turker criteria to find better work on MTurk. Novices in our study viewed over 25,000 tasks by 1,394 requesters. We found that novices who utilized this Super Turkers' criteria earned better wages than other novices. Our results highlight that tool development to support crowd workers should be paired with educational opportunities that teach workers how to effectively use the tools and their related metrics (e.g., transparency values). We finish with design recommendations for empowering crowd workers to earn higher salaries.

Introduction: The Guidance Gap in Crowd Work

Crowd markets like Amazon Mechanical Turk (MTurk) are notorious for "invisible labor" and low pay, with many workers struggling to reach the U.S. minimum wage of $7.25/hour. While the community has developed transparency tools like Turkopticon and TurkerView to rate requesters, having data isn't the same as having a strategy. Novices often feel overwhelmed by the various metrics—generosity, promptness, fairness—and end up picking tasks inefficiently. This paper addresses the "guidance gap" by identifying the heuristics used by the platform’s top earners—the "Super Turkers"—and testing if those strategies can be successfully transferred to beginners.

Transparency tools like TurkerView and Turkopticon provide critical requester data directly within the MTurk interface.

Methodology

The researchers employed a two-phase approach to bridge this gap:

  1. Strategy Extraction: They surveyed 100 Super Turkers (defined as having completed >10,000 tasks and earning above minimum wage). Using the Analytic Hierarchy Process (AHP) and a Longest Common Subsequence (LCS) algorithm, they distilled the experts' complex workflows into a simple, three-step hierarchical decision tree.
  2. Field Experiment: They recruited 100 novice workers and conducted a 12-day study using a custom Chrome plugin. The plugin tracked "real" hourly wages by measuring the exact time spent searching for and completing tasks. They split the novices into a control group (using tools as they saw fit) and an experimental group (instructed to follow the Super Turker strategy).

The "Super Turker" Heuristic

The study revealed a clear priority list for selecting work. High earners do not weigh all metrics equally; they prioritize immediate financial return and safety from rejection. The distilled strategy follows this hierarchy:

  1. Primary Filter: Prioritize tasks where TurkerView predicts an hourly wage >$8.29/h.
  2. Secondary Filter: If wage data is missing, only work for requesters with a Turkopticon fairness score >3.69.
  3. Fallback: If no reputation data exists, only accept tasks with a flat reward >$0.23.
Super Turkers prioritize hourly pay and pay sentiment over other metrics like communication or approval time.

Key Results and Insights

Higher Earnings and Resilience

The experimental group saw significant gains. Individual "Super HITs" identified by the criteria paid a median of $7.04/h**, compared to just **$3.27/h for non-Super HITs. Even after accounting for the extra unpaid time spent searching for better work, the median hourly wage for those following the strategy was $4.12/h**, nearly double the **$2.13/h earned by the control group. Furthermore, when the general pool of available tasks decreased in quality during the study, the novices following the Super Turker strategy maintained their wage levels while the control group's wages plummeted.

The Search Time Trade-off

Identifying high-quality work is not free. Novices spent a median of 61 seconds searching for a "Super HIT" compared to just 9 seconds for a standard task. This highlights that higher wages require a deliberate investment in search time—a behavior common among experts but often overlooked by novices who rush into low-paying tasks.

Fairness as a Proxy

When wage data is unavailable, Super Turkers rely heavily on "Fairness" ratings. This serves as a defensive strategy to avoid "algorithmic cruelty," where a requester might arbitrarily reject work, resulting in zero pay and a damaged worker reputation.

Conclusion and Future Work

The study demonstrates that tool development must be paired with education. Simply providing transparency data is insufficient; workers need actionable heuristics to navigate the market effectively. Future work should explore "on-the-job tutorials" or more assertive interfaces to guide novice behavior, helping them overcome the compliance hurdles and qualification barriers that currently limit their earning potential.

Summary updated

Abstract

Crowd markets have traditionally limited workers by not providing transparency information concerning which tasks pay fairly or which requesters are unreliable. Researchers believe that a key reason why crowd workers earn low wages is due to this lack of transparency. As a result, tools have been developed to provide more transparency within crowd markets to help workers. However, while most workers use these tools, they still earn less than minimum wage. We argue that the missing element is guidance on how to use transparency information. In this paper, we explore how novice workers can improve their earnings by following the transparency criteria of Super Turkers, i.e., crowd workers who earn higher salaries on Amazon Mechanical Turk (MTurk). We believe that Super Turkers have developed effective processes for using transparency information. Therefore, by having novices follow a Super Turker criteria (one that is simple and popular among Super Turkers), we can help novices increase their wages. For this purpose, we: (i) conducted a survey and data analysis to computationally identify a simple yet common criteria that Super Turkers use for handling transparency tools; (ii) deployed a two-week field experiment with novices who followed this Super Turker criteria to find better work on MTurk. Novices in our study viewed over 25,000 tasks by 1,394 requesters. We found that novices who utilized this Super Turkers' criteria earned better wages than other novices. Our results highlight that tool development to support crowd workers should be paired with educational opportunities that teach workers how to effectively use the tools and their related metrics (e.g., transparency values). We finish with design recommendations for empowering crowd workers to earn higher salaries.
BibTeX
@inproceedings{savage2020becoming,
  author = {Saiph Savage and Chun-Wei Chiang and Susumu Saito and Carlos Toxtli and Jeffrey P. Bigham},
  title = {{Becoming the Super Turker: Increasing Wages via a Strategy from High Earning Workers}},
  booktitle = {The Web Conference},
  year = {2020},
  doi = {10.1145/3366423.3380200},
}
Figure