Aggressive language in an online hacking forum

Andrew Caines, Sergio Pastrana, Alice Hutchings, Paula ButteryEMNLP 2018

TL;DR

This paper investigates whether "aggressive language" is a monolithic concept or varies across online domains. By training a classifier on Wikipedia edit comments and testing it on a hacking forum, they found that aggressive language in hacking forums is generally milder and more focused on posturing than hate speech or threats, suggesting domain-specific nuances in online aggression.

What Problem Does This Paper Address?

The automatic detection of abusive language online is a growing area of research with significant social implications. However, much of this research assumes a universal definition of "aggressive language." This paper questions that assumption, asking whether the nature and intensity of aggressive language differ across distinct online communities. Specifically, they compare aggressive language found in Wikipedia edit comments, which are often adversarial, with that found in an online hacking forum, aiming to understand if a classifier trained on one domain can effectively identify aggression in another, and what insights this comparison yields about the nature of online aggression itself.

Methods

The authors employ a standard document classification approach. They use an extreme gradient boosting (XGBoost) classifier, a robust and widely used algorithm for such tasks. The core of their methodology lies in an inter-corpus experimental design: they train a binary aggression classifier on a large, annotated dataset of Wikipedia comments and then test its performance on a manually labeled dataset from a hacking forum. They experiment with different thresholds for defining "aggressive" posts in the training data, systematically varying the cut-off point for attack scores to see how it impacts classification accuracy.

Datasets

  • Training Data: Wikipedia Comments Corpus (WikiComments). This corpus contains over 115,000 discussion posts from Wikipedia edits, annotated for personal attacks or harassment with an "attack score" from 0 to 10.
  • Test Data: CrimeBB Corpus (specifically, a subset from HackForums). This dataset consists of 4,123 manually labeled posts from a hacking forum, specifically from boards related to cryptography, encryption, keyloggers, and remote administration tools. The authors also note that the CrimeBB Corpus itself is much larger and contains data from other hacking-related websites.

Key Insights and Ideas

  • Domain Matters: The nature of aggressive language is not uniform across the internet. Hacking forums, despite their potentially confrontational nature, exhibit milder forms of aggression compared to the more adversarial environment of Wikipedia edit comments.
  • Purpose Influences Aggression: The primary purpose of interactions in a domain significantly shapes the manifestation of aggression. Hacking forums often serve constructive purposes (education, information exchange), which may act as a natural deterrent to extreme aggression. Wikipedia edit comments, conversely, are geared towards disputes and adversarial interactions.
  • Distinction Between Offensive and Aggressive: The paper emphasizes the difference between merely offensive language (e.g., profanity) and genuinely aggressive language, which involves a hostile stance. Profanity can be used humorously or casually, whereas aggression is about intent and tone.
  • Reputation Systems as Deterrents: In the context of the hacking forum, the presence of an active reputation scoring system is suggested as a mechanism that may deter aggressive online harassment and foster trust.

Key Results

  • A classifier trained on Wikipedia comments could identify aggressive posts in the hacking forum, but the optimal threshold for defining aggression in the training data differed significantly. For Wikipedia, the optimal threshold was found to be 4.25, while for the hacking forum, the optimal threshold was between 2 and 3. This suggests that what constitutes "aggressive" in the Wikipedia context is more severe than in the hacking forum context.
  • Aggressive language in the hacking forum subset of CrimeBB was found to be relatively mild. The most common types were "simply aggressive" (48% of aggressive posts) and "personal denigration" (37%), often involving questioning intelligence or motives.
  • Direct threats of violence were rare in the hacking forum subset (only 2 examples), and hate speech (discriminating against groups) was also infrequent.
  • Profane language was common in the hacking forum, but often used humorously or casually, underscoring the distinction between offensive and aggressive language.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that the number of aggressive posts in their hacking forum dataset (100 out of 4123) is relatively small, which can limit the robustness of findings. They suggest investigating automatic annotation of larger chunks of data and supervised sampling to increase the number of labeled abusive examples.
  • Inter-Domain Comparison: While they compared Wikipedia and one hacking forum, they propose investigating other Internet domains, including social media, other forums, and even the Dark Web, to build a more comprehensive understanding of aggressive language across different online environments.
  • Ethical Considerations: The paper touches upon the ethical implications of data collection and intervention, suggesting future work on desired accuracy for deployed systems and appropriate actions for moderators.
  • Classifier Performance: While their XGBoost classifier performed reasonably well, they note that other models, such as neural networks, could be explored in future work. They also mention that their AUC scores were lower than those achieved by some previous studies on Wikipedia data, suggesting room for improvement.

Summary updated

BibTeX
@inproceedings{caines2018aggressive,
  author = {Andrew Caines and Sergio Pastrana and Alice Hutchings and Paula Buttery},
  title = {{Aggressive language in an online hacking forum}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure