Abstract
As human computation on crowdsourcing systems has become popular and powerful for performing tasks, malicious users have started misusing these systems by posting malicious tasks, propagating manipulated contents, and targeting popular web services such as online social networks and search engines. Recently, these malicious users moved to Fiverr, a fast-growing micro-task marketplace, where workers can post crowdturfing tasks (i.e., astroturfing campaigns run by crowd workers) and malicious customers can purchase those tasks for only $5. In this paper, we present a comprehensive analysis of Fiverr. First, we identify the most popular types of crowdturfing tasks found in this marketplace and conduct case studies for these crowdturfing tasks. Then, we build crowdturfing task detection classifiers to filter these tasks and prevent them from becoming active in the marketplace. Our experimental results show that the proposed classification approach effectively detects crowdturfing tasks, achieving 97.35% accuracy. Finally, we analyze the real world impact of crowdturfing tasks by purchasing active Fiverr tasks and quantifying their impact on a target site. As part of this analysis, we show that current security systems inadequately detect crowdsourced manipulation, which confirms the necessity of our proposed crowdturfing task detection approach.
Introduction: The Rise of Crowdturfing
Crowdsourcing platforms like Amazon Mechanical Turk and Fiverr have revolutionized the distribution of micro-tasks to a global workforce. However, these platforms have a significant "dark side": the rise of crowdturfing (crowdsourced astroturfing). Malicious actors leverage these marketplaces to buy and sell fake reviews, artificial social media followers, and manipulated search engine backlinks.
While the existence of crowdturfing has been noted previously, the specific dynamics of the Fiverr marketplace—a platform where every task starts at $5—remained largely unexamined. This research characterizes the Fiverr ecosystem and develops a system to automatically detect and filter these malicious tasks before they can impact the broader web.
Methodology and Data Collection
To analyze the marketplace, the researchers developed a custom crawler to scrape Fiverr, collecting 89,667 active gig listings and 31,021 user profiles between July and August 2013. From this pool, a subset of 1,550 gigs was manually labeled as either "legitimate" or "crowdturfing" (consisting of 121 crowdturfing and 1,429 legitimate tasks).
Using this labeled dataset, the team extracted several features to train a machine learning classifier:
- Textual Features: Gig titles and descriptions processed via TF-IDF bag-of-words.
- Metadata: Categories, seller ratings, and seller demographics (e.g., country of origin, account longevity).
The study evaluated over 30 machine learning algorithms, including Naive Bayes, Support Vector Machines (SVM), and various tree-based models, to identify the most effective detection method.
Characterizing the Marketplace
The analysis revealed several critical insights into the crowdturfing economy:
- High Financial Incentives: Crowdturfing is remarkably lucrative. The top seller in the dataset was estimated to have earned $3 million over two years, primarily through SEO manipulation (backlink) gigs.
- Western Dominance: Unlike many micro-task platforms where workers are concentrated in developing nations, a plurality of Fiverr crowdturfing sellers (35.8%) and buyers are located in the United States. The $5 price point makes it a viable side-hustle for Western users.
- Primary Targets: Manipulation efforts are concentrated in three areas: Social Media (70.7%, e.g., fake likes/followers), Search Engines (27.3%, e.g., artificial backlinks), and Direct Traffic (2%, e.g., bot traffic for ad fraud).
Results and Real-World Impact
The proposed detection system achieved high performance, with the SVM-based classifier reaching 97.35% accuracy (F1 score of 0.974 and a False Positive Rate of 0.008). Applying this classifier to the entire dataset suggested that approximately 22.2% of the active marketplace consisted of crowdturfing tasks.
To test the real-world efficacy of these services, the researchers purchased five Twitter follower gigs for $25. Within 48 hours, they received 46,176 followers, boosting the target accounts' Klout influence scores from 0 to nearly 20. Critically, two months later, Twitter's security systems had suspended only 24.6% of these fake accounts, demonstrating that current platform defenses are largely ineffective against crowdsourced manipulation.
Conclusion
This study highlights the scale and sophistication of the crowdturfing market on Fiverr. The high accuracy of the developed classifier suggests that marketplaces and target platforms could proactively mitigate these threats by integrating automated detection systems into their security backends. Future work should focus on the real-time deployment of these classifiers to prevent malicious tasks from being executed.
Summary updated