TL;DR
This paper introduces a novel WhatsApp dataset of pre-teen cyberbullying interactions, collected through a unique "living lab" role-playing experiment. By overcoming the privacy barriers of instant messaging apps, this work provides a valuable resource for computational analysis of cyberbullying and offers insights into the ethical and operational challenges of such research.
What Problem Does This Paper Address?
Cyberbullying is a significant issue among adolescents, with instant messaging apps like WhatsApp being a primary channel for these interactions. However, the private nature of these platforms and their privacy policies make it extremely difficult to collect data for research. Existing studies often rely on self-reported surveys, which can be prone to biases. This lack of accessible, real-world data hinders the development of computational methods to detect, understand, and ultimately prevent cyberbullying. The authors aim to bridge this gap by creating a rich, annotated dataset of pre-teen cyberbullying conversations.
Methods
The core of this research is the creation of a WhatsApp dataset through a "living lab" methodology. This involved a role-playing experiment with Italian students aged 12-13. The experiment was embedded within a larger project (CREEP) focused on cyberbullying prevention. Key methodological components include:
- Living Lab Approach: This involved close collaboration with schools, teachers, and parents, creating a participatory design process.
- Role-Playing Experiment: Students were assigned roles (cyberbully, victim assistant, bystander-defender) and engaged in simulated WhatsApp conversations based on pre-defined scenarios. Crucially, researchers impersonated the victims to avoid exposing minors to direct harassment.
- Data Collection: WhatsApp chats were collected anonymously from dedicated mobile phones provided by the project.
- Annotation: The collected chats were manually annotated by two expert linguists using a fine-grained annotation scheme adapted from existing guidelines, covering user roles, types of cyberbullying expressions, sarcasm, and joke identification.
Datasets
The paper introduces a newly created WhatsApp dataset of pre-teen cyberbullying interactions.
- Size: The corpus consists of 14,600 tokens across 10 chats.
- Content: It comprises anonymized WhatsApp conversations from a role-playing experiment involving 70 students aged 12-13.
- Annotations: The dataset is annotated with:
- Cyberbullying Roles: Harasser, Victim, Bystander-defender, Bystander-assistant.
- Cyberbullying Types: A detailed taxonomy including threats, general insults, body shame, discrimination, sexual harassment, etc.
- Other Attributes: Presence of sarcasm and whether an expression is a joke.
- Format: Annotated data is available in a standoff XML format.
This dataset differs from existing ones by focusing specifically on WhatsApp, a private messaging platform, and by using a simulated role-playing environment to ethically gather data from minors.
Key Insights and Ideas
- The "Living Lab" as a Bridge: The living lab approach proved effective in fostering trust and ensuring participation, demonstrating its value for sensitive research with minors.
- Role-Playing for Ethical Data Collection: Impersonating victims by researchers was a key strategy to ethically collect data on cyberbullying without directly exposing minors to harassment.
- Fine-Grained Annotation is Crucial: Adapting and extending existing annotation guidelines allowed for a detailed understanding of the nuances of pre-teen cyberbullying language.
- Body Shame as a Significant Issue: The inclusion of "Body Shame" as a distinct annotation category reflects its growing importance in adolescent cyberbullying.
- Bystander Roles are Active: The data shows a significant number of "Bystander-defender" interactions, highlighting the active role of peers in cyberbullying dynamics.
Key Results
- Prevalence of Cyberbullying Expressions: Cyberbullying expressions constituted 41.1% of the entire corpus, indicating the phenomenon's significant presence even in a simulated environment.
- Balanced Roles: The distribution of cyberbullying roles among participants was relatively balanced.
- Defense Expressions Dominate: Expressions of "Defense" (from victims or bystander-defenders) were the most numerous (31.7%), often including support for the victim or disapproval of the harassment.
- Absence of Certain Insult Types: Undesirable sexual talk and discrimination based on race, skin color, ethnicity, nationality, or religion were not found in the corpus.
- Inter-Annotator Agreement: Satisfactory agreement (Dice coefficient ≥ 0.80 for extension and accuracy for attributes) was achieved between annotators, validating the annotation guidelines.
- Qualitative Findings: Scenarios related to ballet and viral videos elicited specific types of insults (sexist nature, body shame), demonstrating how context influences cyberbullying expressions.
Limitations and Future Work
- Realism of Simulated Data: A primary limitation acknowledged is the potential difference between simulated interactions and actual cyberbullying. The authors tentatively assessed plausibility through observer effect, interaction evolution, and engagement, suggesting the spirit of the game might lead to realistic behavior. However, a robust methodology for evaluating this is still needed.
- Exclusion of Students without Smartphones: Students without personal smartphones were excluded, raising concerns about participation bias. The decision to prioritize realism over universal participation is noted.
- Ethical Considerations: Exposing pre-teens to offensive language, even in a simulated environment, remains an ethical challenge.
- Future Work: The authors suggest the need for further research into evaluating the realism of the collected corpus and developing methodologies to assess it. They also highlight the potential of this dataset for studying code-switching and the impact of informal language on writing skills. The living lab approach is also proposed as a model for future research in similar domains.
Summary updated