Anyone Can Become a Troll

Michael S. Bernstein, Justin Cheng, Cristian Danescu-Niculescu-Mizil, Jure LeskovecAmerican 2017 · DOI

TL;DR

This paper challenges the notion that trolls are born, not made, by demonstrating that ordinary individuals can engage in trolling behavior due to situational factors like negative mood and the discussion context. Through a controlled experiment and a large-scale analysis of CNN.com discussions, they show that these factors significantly increase the likelihood of trolling, suggesting that online communities can be designed to mitigate this behavior.

What Problem Does This Paper Address?

Online discussions are increasingly prevalent, but antisocial behavior like trolling significantly disrupts constructive dialogue. While prior work often attributes trolling to a small group of inherently antisocial individuals (trolls), this paper questions whether ordinary people can also engage in such behavior under certain circumstances. They aim to understand the situational triggers that lead to trolling and how it might spread within online communities, moving beyond the "trolls are born, not made" paradigm.

Methods

The researchers employed a two-pronged approach:

  1. Online Experiment: They conducted a controlled experiment simulating an online news discussion. Participants were exposed to either positive or negative stimuli designed to induce a specific mood (PosMOOD/NEGMOOD) and were then placed in discussion threads where initial posts were either benign (PosCONTEXT) or troll-like (NEGCONTEXT). This allowed them to isolate the causal effects of mood and discussion context on trolling behavior.
  2. Large-Scale Data Analysis: They analyzed over 16 million posts from CNN.com, a popular online news discussion platform. This observational study allowed them to:
  3. Investigate how population-level mood shifts (inferred from time of day and day of week) correlate with trolling behavior.
  4. Examine the persistence of trolling behavior across multiple discussions.
  5. Analyze the impact of discussion context, specifically the presence, volume, and ordering of prior troll posts.
  6. Develop a predictive model of trolling behavior using various features.

Datasets

  • Experimental Data: Collected from participants on Amazon Mechanical Turk (AMT) who engaged in simulated online discussions.
  • CNN.com Discussion Data: A large dataset comprising over 16 million posts from December 2012 to August 2013, including user activity, discussions, flagged posts, and deleted posts.

Key Insights and Ideas

  • Situational Triggers are Key: The most striking insight is that negative mood and exposure to prior trolling significantly increase the probability of an ordinary user trolling. This directly challenges the idea that trolling is solely an innate personality trait.
  • Trolling is Contagious: The "Broken Windows" hypothesis seems to apply to online discussions; untended negative behavior (trolling) can lead to a breakdown in civility and encourage more of the same.
  • Mood Fluctuations Mirror Trolling Patterns: Trolling behavior on CNN.com mirrors daily and weekly mood patterns, peaking during evenings and at the start of the week, suggesting a link between collective mood and online aggression.
  • Discussion Context Matters More Than Innate Propensity: A predictive model showed that discussion context features (like prior troll posts) were more informative for predicting trolling than individual user history or inferred innate propensity.
  • Negative Mood Lingers: The effects of negative mood and exposure to trolling can persist across discussions, indicating that the impact of a bad experience isn't confined to a single interaction.

Key Results

  • Experimental Findings:
    • Negative mood (NEGMOOD) increased the probability of trolling by 89%.
  • The presence of prior troll posts (NEGCONTEXT) increased the probability of trolling by 68%.
  • Combining negative mood and troll context doubled participants' baseline trolling rates.
  • CNN.com Analysis:
    • 40% of flagged posts were authored by users with no prior record of such behavior, supporting the idea that ordinary users troll.
  • Trolling behavior (inferred from flagged posts) peaked in the evenings and on Mondays, correlating with known mood patterns.
  • Participating in a discussion with at least one flagged post made users significantly more likely to troll in subsequent, unrelated discussions.
  • The probability of a post being flagged increased monotonically with the number of prior flagged posts in a discussion.
  • A predictive model using discussion context features achieved an AUC of 0.74, outperforming models based solely on user history or mood proxies.

Limitations and Future Work

  • Limitations:
    • Mood Measurement: In the large-scale analysis, mood was inferred indirectly through temporal patterns (time of day, day of week) and user history, rather than direct measurement.
  • Experimental Incentives: Participants in the experiment were compensated, which might influence their behavior and incentives.
  • Anonymity: The use of pseudonyms in the experiment could have affected comment quality.
  • Article Topic: While article topic was considered, its interplay with other factors like personal values and group membership requires further exploration.
  • Future Work:
    • Develop more nuanced mood signals using behavioral traces.
  • Incorporate discussion structure (reply chains, sentiment flow) into models.
  • Differentiate between various trolling strategies and their prevalence.
  • Investigate how social cues (e.g., anonymity vs. real names) mediate trolling.
  • Distinguish between users who troll due to innate traits versus situational factors.
  • Explore how to design platforms that mitigate situational trolling factors, rather than solely relying on banning users.

Summary updated

BibTeX
@inproceedings{bernstein2017anyone,
  author = {Michael S. Bernstein and Justin Cheng and Cristian Danescu-Niculescu-Mizil and Jure Leskovec},
  title = {{Anyone Can Become a Troll}},
  booktitle = {American},
  year = {2017},
  doi = {10.1511/2017.105.3.152},
}
Figure