Classification of Adolescents' Risky Behavior in Instant Messaging Conversations

Jaromír Plhák, Ondrej Sotolár, Michaela Lebedíková, David SmahelAISTATS 2023

This research, originally presented at AISTATS 2023, offers a rare look into the "digital wild west" of private adolescent communication. While most research on online risk focuses on public platforms like Twitter or YouTube, these authors tackle the much harder—and more relevant—domain of private instant messaging (IM).

The Challenge: Private vs. Public Data

Adolescents spend a massive portion of their social lives in private IM apps like WhatsApp and Messenger. These spaces are prone to risks like cyber-aggression, discussions of self-harm, and substance abuse. However, detecting these risks is difficult for two primary reasons:

  1. The Data Gap: Most existing models are trained on public social media data. The language used in a public tweet is fundamentally different from a private, slang-heavy chat between two 14-year-olds.
  2. The Needle in the Haystack: Even though these risks are serious, they occur relatively rarely in the total volume of messages, making it hard for standard algorithms to "learn" what they look like without being overwhelmed by "normal" chatter.

Methodology: Transformers and Context

The authors employed a supervised learning approach using Transformer architectures, which are currently the state-of-the-art for understanding text in context. Key technical strategies included:

  • Model Comparison: They compared three models: a small monolingual Czech model (Small-E-Czech), a full-size monolingual model (RobeCzech), and a multilingual model (XLM-RoBERTa).
  • Context Windows: Unlike simple text classifiers that look at one sentence at a time, the authors experimented with "context length"—prepending previous messages (up to 240 characters) to the target message to see if the surrounding conversation helped the model understand intent.
  • Data Augmentation: To combat the scarcity of "risky" messages, they used back-translation. They translated Czech messages into English and German and then back to Czech to create synthetic "paraphrased" training examples.

Dataset: A Unique Corpus of Adolescent Chat

The study used a unique, proprietary corpus of 1.26 million utterances from 2,165 Czech adolescents, manually exported from Facebook Messenger. The data underwent rigorous multi-level anonymization, replacing names and media with tokens to protect participant privacy.

Treemaps showing the distribution of annotated utterances across the four risk categories: (a) Aggression, (b) Mental health, (c) Alcohol/drugs, and (d) Sexual content.

A subset of ~14,000 utterances was meticulously annotated by trained researchers and supervised by social scientists to identify four specific risks: (1) Aggression/Hate, (2) Mental Health, (3) Alcohol/Drugs, and (4) Sexual Content. The figure above visualizes the distribution of these annotated samples, highlighting the class imbalance inherent in the data.

Key Insights and Ideas

  • Monolingual > Multilingual: The monolingual RobeCzech model consistently outperformed the much larger multilingual XLM-RoBERTa. This suggests that for niche, slang-heavy domains, language-specific nuances are more important than sheer model size.
  • Context is Category-Dependent: Adding conversation context significantly improved the detection of alcohol and drug references. However, for aggression, context had a "sweet spot" (around 90 characters) beyond which performance actually degraded, likely because too much history introduces noise.
  • The Sarcasm Trap: Error analysis showed that the model struggles with sarcasm and "inter-group language style." For example, friends calling each other names in a joking way were often flagged as "aggression," even when the human context suggested otherwise.
  • Augmentation Limits: Surprisingly, back-translation didn't significantly boost performance. The authors suggest that the model needs more diverse examples of risk, not just reworded versions of the same few examples.

Results and Performance

The models achieved promising, though varied, results across the four categories:

  • Sexual Content: Achieved the highest F1 score of .828.
  • Mental Health: Achieved an F1 of .66.
  • Alcohol/Drugs: Achieved an F1 of .654.
  • Aggression/Hate: Proved the most difficult, with an F1 of .493.

The authors note that the high score for sexual content might be slightly inflated because that data came from a smaller number of users, meaning the model might have partially learned those specific individuals' writing styles.

Limitations and Future Work

The biggest limitation is reproducibility. Because the dataset contains highly sensitive private conversations, it cannot be released to the public, making it hard for other researchers to verify or build upon the work directly. Furthermore, the "Mental Health" and "Sexual Content" categories suffered from low data variance—a few very active users provided the bulk of the positive examples. Future work will explore mT5 models for better paraphrase generation to solve the class imbalance problem more effectively.

Summary updated

BibTeX
@inproceedings{plhak2023classification,
  author = {Jaromír Plhák and Ondrej Sotolár and Michaela Lebedíková and David Smahel},
  title = {{Classification of Adolescents' Risky Behavior in Instant Messaging Conversations}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2023},
}
Figure