Abstract
BiRDy is a novel web-based platform presented at AAAI-23 for detecting participant roles in cyberbullying episodes within private, multi-party chats. By leveraging fine-tuned multilingual BERT (mBERT) and a contextual window strategy, BiRDy identifies roles such as "bully," "victim," and "defender." The system addresses the unique challenges of private messaging, including data privacy and conversation entanglement, achieving high F-scores in both French and Italian datasets.
Introduction
While cyberbullying research often focuses on public platforms like Twitter or Reddit, a significant portion of teen aggression occurs in private instant messaging (IM) apps like WhatsApp or Messenger. These environments pose two primary challenges: the difficulty of collecting private data and the "semi-asynchronous" nature of chat logs, where multiple conversation threads overlap. BiRDy aims to overcome these hurdles by focusing on textual content and temporal structure to map participant dynamics.
Methods
The BiRDy system utilizes a two-stage pipeline for role detection:
- Message Labeling: Individual messages are classified using a fine-tuned mBERT model. To handle conversation entanglement, a contextual window strategy is employed, where the model considers the $n$ previous messages from the same user to capture the narrative arc.
- Role Assignment: A role scoring function aggregates message probabilities to assign each participant a single primary role (e.g., Bully, Victim, Defender, or Bystander Assistant).
The study utilized two datasets created through role-playing simulations: CyberAgressionAdo-v1 (French, 2,921 entries) and an Italian WhatsApp-style dataset (2,114 entries). These datasets include manual annotations for nuanced roles beyond simple binary classification.
Results and Discussion
Experimental results demonstrate that context is vital for accurate detection. Performance generally improves as the contextual window size increases, peaking at a window of 9 for French and 10 for Italian.
The model achieved weighted F-scores of 0.778 for French and 0.869 for Italian. However, performance varied significantly by role and language. In the Italian dataset, bullies were identified with high accuracy (reaching an F-score of 1.0 in some settings), whereas victims were more easily identified in the French dataset.
These variations suggest that the linguistic "scripts" of bullying and victimization may differ across cultures or specific training scenarios. Misclassifications typically occurred between linguistically similar roles, such as "bullies" and "bystander assistants."
Conclusion
BiRDy provides a robust framework for understanding the dynamics of private group chats. Future work will address the fluidity of roles over time, expand the system to more languages, and evaluate the model on larger, more diverse datasets.
Demo available at: http://134.59.134.227/demo_prd/index.html
Summary updated