Abstract
In online labor markets, task design often involves sequences of work rather than isolated tasks. This study investigates how the ordering of task price and difficulty affects worker performance. Through experiments on Amazon Mechanical Turk, we find that initial price anchors significantly influence subsequent work quality, whereas transitions between difficulty levels have surprisingly little impact. These findings suggest that workers psychologically adjust their effort and expectations based on the sequence's starting point and task demands.
Introduction
Online crowdsourcing platforms like Amazon Mechanical Turk (MTurk) allow requesters to decompose complex workflows into sequences of Human Intelligence Tasks (HITs). While previous research has explored the impact of pay on performance, the longitudinal effects of task sequences—specifically how early experiences "anchor" later performance—remain under-explored. This paper addresses two primary questions: how long do initial price anchors persist in a sequence, and how do transitions between varying difficulty levels affect output quality?
Experimental Design
The researchers conducted two sets of experiments using three task types: Button Clicking (BC) for motor skills, Spotting Difference (SD) for visual attention, and Spatial N-back (NB) for cognitive load.
Experiment 1: Price Anchoring A three-task sequence was used with two price levels: low (4¢) and high (32¢). Five sequences were tested (e.g., 4-4-4, 32-4-4, 4-32-4) to see how the history of pay influenced the quality of the final task.
Experiment 2: Difficulty Transitions A two-task sequence was used where pay remained constant, but difficulty varied across four levels (Easy to Very Hard). This aimed to identify "switch costs" or fatigue effects when moving between tasks of different intensities.
Results: The Power of Price Anchoring
The results from Experiment 1 demonstrate a strong "first impression" effect. The price of the first task in a sequence exerts a more significant influence on final work quality than the price of the second task.
As shown in Figure 4, workers in the 32–4–4 sequence (starting high and dropping) performed significantly worse on the final task than those who started and stayed at the low price (4–4–4). Conversely, those in the 4–32–4 sequence outperformed the 32–4–4 group on the final task, despite both ending at the same price point. This confirms that the sequence of pay is as critical as the absolute amount.
Results: Difficulty Invariance
Contrary to standard psychological predictions of "switch costs" or resource depletion, shifting between easy and hard tasks did not significantly impact quality.
In the Button Clicking task (Figure 3), workers transitioning from a "Hard" task to an "Easy" task performed almost identically to those who performed two consecutive "Easy" tasks. Post-task surveys suggest an "anchoring on effort" mechanism: workers adjust their perception of "fair pay" based on task difficulty. This psychological recalibration appears to offset potential physical fatigue or cognitive switching costs, resulting in stable performance across transitions.
Discussion and Conclusion
The study highlights that requesters must be cautious when designing task sequences. A high initial price can set an unsustainable anchor, leading to quality degradation if subsequent tasks pay less. However, the lack of a difficulty effect suggests that requesters have more flexibility in ordering tasks of varying complexity, provided the perceived effort-to-pay ratio remains acceptable to the worker.
Future work should investigate whether these anchoring effects persist over much longer sequences (e.g., 20+ tasks) and whether the difficulty invariance holds when switching between entirely different task categories (e.g., from image labeling to transcription).
Summary updated