Forecasting the Presence and Intensity of Hostility on Instagram Using Linguistic and Social Features

Ping Liu, Joshua Guberman, Libby Hemphill, Aron Culotta

Abstract

Online antisocial behavior, such as cyberbullying and harassment, threatens free discussion and has severe mental health consequences. While automated methods exist to identify hostile comments, they typically work retrospectively. This paper proposes a proactive approach: forecasting future hostilities. We decompose this into two tasks: (1) predicting if a hostile comment will appear given a sequence of clean comments, and (2) predicting if an initial hostile comment will lead to an escalation. Using a new corpus of 30K annotated Instagram comments, our approach achieves an AUC of 0.82 for predicting hostility presence ten hours in advance and an AUC of 0.91 for distinguishing between high and low levels of future hostility.

Introduction: From Detection to Forecasting

Most current automated moderation tools operate in a "reactive" loop. They identify harassment or hate speech only after it has been posted and viewed, meaning victims are already exposed to toxic content before intervention occurs. By the time a system flags a comment, the interaction has often already escalated.

This research shifts the focus toward proactive forecasting. By treating hostility as a predictable event rather than a random occurrence, moderators can potentially intervene before a conflict begins. The study defines two primary forecasting challenges:

  1. Hostility Presence Forecasting: Predicting whether a currently "clean" thread will attract a hostile comment in the future.
  2. Hostility Intensity Forecasting: Predicting whether a single hostile comment will devolve into a high-volume "conflict spiral."

Methodology and Feature Engineering

The researchers utilized a Logistic Regression model with L2 regularization. To ensure interpretability and robustness, they focused on sophisticated feature engineering rather than complex neural architectures. The feature set included:

  • Linguistic Features: Standard unigrams and Word2vec embeddings, including n-gram Word2vec to account for the typos and abbreviations prevalent in social media text.
  • Lexical Signals: Features derived from established databases like "Hatebase" and offensive word lists.
  • Social and Contextual Features: The model analyzed user activity (ratio of unique users to total comments) and trend features (the trajectory of hostility probability within a specific thread).
  • Historical Context: A key innovation was tracking the recent behavior of the post's author and current commenters in other threads to determine if the general "vibe" of their recent interactions predicted future conflict.

Dataset Characteristics

The study introduced a manually annotated corpus of 30,987 Instagram comments across 1,134 posts. The sampling strategy targeted the teenager demographic, as they are particularly vulnerable to cyberbullying. To handle the natural class imbalance of social media (where most comments are benign), the researchers used keyword-based snowball sampling to identify potentially hostile threads, balancing them with non-hostile posts from similar populations.

Distribution of post characteristics including comment volume, user participation, and hostility timing.

As shown in the data distribution above, the dataset captures a wide range of engagement levels. While the median post receives 14 comments, the distribution is long-tailed. Notably, the median time until the first hostile comment appears is only one day, though the mean is 13 days, suggesting that while some conflicts ignite quickly, others simmer over longer periods.

Key Insights and Results

The model demonstrated strong predictive power across both tasks:

  • Task 1 (Presence): The system achieved an AUC of 0.82 with a lead time of 10 hours. This indicates that a future fight can be reliably predicted nearly half a day before it begins.
  • Task 2 (Intensity): Predicting whether a post would receive ten or more hostile comments was even more successful, reaching an AUC of 0.91. This suggests that once a conflict starts, it follows a highly predictable social pattern.

The "Outsider" and "Target" Effects

One of the most interesting findings was the role of user history. Features derived from the history of current commenters actually reduced accuracy for predicting the start of a conflict. This is because the person who initiates hostility is often an "outsider" who has not yet participated in the thread.

Conversely, the history of the post's author was highly predictive. Certain users consistently receive more hostility based on their past interactions or the nature of their content, making the author's profile a strong signal for potential future harassment.

Limitations and Future Directions

While the results are promising, the researchers noted several areas for growth. The dataset's reliance on keyword seeding may miss subtle, coded, or sarcastic harassment that avoids profanity. Additionally, as Instagram is a visual platform, future models should incorporate image analysis to see if specific visual content triggers hostile responses.

Finally, the transition from forecasting to prevention remains an open question. Future research must explore how "soft" interventions—such as slowing comment frequency or alerting authors—can effectively disrupt the predicted escalation of hostility.

Summary updated

Abstract

Online antisocial behavior, such as cyberbullying, harassment, and trolling, is a widespread problem that threatens free discussion and has negative physical and mental health consequences for victims and communities. While prior work has proposed automated methods to identify hostile comments in online discussions, these methods work retrospectively on comments that have already been posted, making it difficult to intervene before an interaction escalates. In this paper we instead consider the problem of forecasting future hostilities in online discussions, which we decompose into two tasks: (1) given an initial sequence of non-hostile comments in a discussion, predict whether some future comment will contain hostility; and (2) given the first hostile comment in a discussion, predict whether this will lead to an escalation of hostility in subsequent comments. Thus, we aim to forecast both the presence and intensity of hostile comments based on linguistic and social features from earlier comments. To evaluate our approach, we introduce a corpus of over 30K annotated Instagram comments from over 1,100 posts. Our approach is able to predict the appearance of a hostile comment on an Instagram post ten or more hours in the future with an AUC of .82 (task 1), and can furthermore distinguish between high and low levels of future hostility with an AUC of .91 (task 2).
BibTeX
@inproceedings{liu2018forecasting,
  author = {Ping Liu and Joshua Guberman and Libby Hemphill and Aron Culotta},
  title = {{Forecasting the Presence and Intensity of Hostility on Instagram Using Linguistic and Social Features}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/icwsm.v12i1.15022},
}
Figure