Abstract
Traditional crowdsourcing models often treat workers as "stateless" entities, assuming they perform independent micro-tasks without prior context. This paper challenges that assumption by investigating "crowd memory"—the capacity for a collective to retain individual knowledge and pass it to new members. Through experiments involving grid-based search and real-time maze navigation, the authors demonstrate that crowds can function as a single, knowledgeable agent capable of maintaining high performance despite high worker turnover.
Introduction
Most crowdsourcing algorithms are designed for simple, independent tasks where workers arrive, complete a task, and leave without needing context. However, this limits the crowd's potential for complex, continuous tasks like technical support or strategic navigation. This research explores whether crowds can maintain "memory" at two levels: individual retention (returning workers remembering past solutions) and collective learning (the group maintaining knowledge as members are replaced).
Methods
The researchers conducted two sets of experiments to test these hypotheses:
- Individual Memory Retention: Using Amazon Mechanical Turk, workers were tasked with finding "gold" behind one of nine doors in a 3x3 grid. Performance was measured by the number of clicks required. Workers were re-recruited at intervals up to 12 hours to measure how much of the learned solution they retained.
- Collective Learning in Legion: Using the Legion system, a crowd controlled a single character in a Half-Life 2 maze. The task required navigating to and pressing specific buttons. To test implicit knowledge transfer, the researchers relied on "demonstration points"—newcomers could observe the collective's current voting patterns and actions to infer the correct strategy.
Results
Individual Learning and Efficiency
Workers showed a clear ability to learn and remember task solutions. In the door-opening experiment, the average number of attempts required to find the target dropped significantly after the first trial.
As seen in Figure 1, workers moved from a median of 12.16 clicks to just 3.64 by their third attempt. Remarkably, workers returning after a 6-hour break retained 99% of this performance improvement, proving that crowd workers are not merely "cogs" but learners who retain task-specific expertise.
Collective Accuracy and Turnover
The crowd demonstrated a robust ability to maintain accuracy as a single agent. In the continuous navigation task, the collective reached and maintained high accuracy levels despite complete turnovers of the worker pool.
As shown in Figure 3, the percentage of correct actions reached 100% by the fifth round. This "collective memory" is maintained through the overlap of worker generations; as long as some experienced workers remain to demonstrate the correct action, new workers can learn the strategy implicitly.
Discussion and Limitations
The study concludes that a crowd can be more reliable than a single expert because it can remain active indefinitely through a recruitment pipeline. However, this collective memory is fragile. If the number of active workers drops to zero, the "chain of demonstration" is broken, and the memory is lost.
Future work could enhance this memory by introducing explicit communication channels, such as history logs or chat systems, allowing the crowd to handle more nuanced, non-repetitive tasks. Additionally, while individual retention is high, current marketplaces are not optimized for re-recruiting specific experienced workers, representing a missed opportunity for efficiency in large-scale crowd systems.
Summary updated