TL;DR
This paper investigates the effectiveness of five leading Large Language Models (LLMs) in detecting anti-LGBTQIA+ hate speech in user-generated content. While LLMs show promise, their performance varies significantly, and all exhibit overconfidence, highlighting the need for improved calibration and nuanced understanding for truly safe online spaces.
What Problem Does This Paper Address?
The proliferation of user-generated content on online social networks (OSNs) has created an urgent need for robust content moderation to ensure online safety and inclusivity. Anti-LGBTQIA+ hate speech and cyberbullying are particularly pervasive and harmful, leading to severe psychological distress and increased risk of suicidal thoughts and behaviors for affected individuals. While AI, and specifically LLMs, are increasingly employed for content moderation, there's a critical gap in evaluating their effectiveness in detecting anti-LGBTQIA+ content. Existing research often overlooks the specific challenges and nuances of this marginalized group, potentially leading to biased or ineffective moderation. This paper aims to bridge this gap by systematically evaluating several state-of-the-art LLMs on their ability to identify anti-LGBTQIA+ user-generated texts.
Methods
The researchers employed a "zero-shot" approach for LLM evaluation, meaning the models were tasked with classifying text without being explicitly trained on examples of anti-LGBTQIA+ content. Instead, they relied on a carefully crafted prompt that instructed the LLMs to act as moderators and classify text as either "Normal (0)" or "Anti-LGBTIQA+ (1)," also providing a confidence score. Five LLMs were evaluated: DeepSeek-V3, GPT-4o, GPT-4o mini, GPT-01-mini, and Llama3.3-70B. The evaluation involved processing a dataset of user-generated comments and analyzing various performance metrics, including Accuracy (Acc), True Positive Rate (TPR), False Positive Rate (FPR), Precision (Prec), F1-score (F1), and Brier score for calibration.
Datasets
The study utilized a subset of a dataset originally compiled by Pratik et al. (2022), which consists of social media comments from YouTube, Reddit, and X (formerly Twitter). This subset was specifically curated to focus on content relevant to anti-LGBTQIA+ research. After preprocessing and removing common stop words, the dataset comprised 1256 samples labeled as anti-LGBTQIA+ and 50243 samples labeled as non-anti-LGBTQIA+.
Key Insights and Ideas
- LLMs are not a silver bullet: While LLMs show potential, their performance is not uniform, and significant limitations remain, especially concerning calibration.
- Overconfidence is a major issue: All tested LLMs tend to be overconfident in their predictions, meaning their reported confidence scores are often higher than the actual likelihood of their classification being correct. This can lead to unreliable moderation.
- Nuance is hard: LLMs struggle to distinguish between genuine discussions about LGBTQIA+ topics and harmful rhetoric, often misclassifying texts that contain identity-related terms alongside offensive language.
- Bias lurks in the data: The misclassifications highlight how LLMs can pick up on sociocultural biases present in the training data, leading to difficulties in contextual understanding.
Key Results
- DeepSeek-V3 and Llama3.3-70B led the pack: DeepSeek-V3 achieved the highest Accuracy (0.866) and F1-score (0.871), while Llama3.3-70B boasted the highest Precision (0.891). Both models also had the lowest False Positive Rate.
- GPT-4o mini and GPT-4o showed better calibration: Despite lower overall performance metrics, these models had the lowest Brier scores (0.551 and 0.585, respectively), indicating better probabilistic calibration.
- GPT-01-mini underperformed significantly: This model showed limitations across all evaluation metrics compared to the others.
- High Brier scores across the board: All LLMs exhibited Brier scores above 0.5, indicating a general need for improved probability calibration.
- Common offensive terms: The analysis revealed frequent use of slurs like "faggot," "tranny," and "queer," alongside profanity and aggressive language in anti-LGBTQIA+ texts.
Limitations and Future Work
The authors acknowledge that while LLMs show promise, their current performance, particularly regarding calibration, is not sufficient for real-world deployment without further improvement. They suggest that fine-tuning LLMs, utilizing larger and more diverse datasets, and incorporating strategies like few-shot prompting could significantly enhance their predictive capabilities. Future work should also focus on ensuring demographic representation in datasets to assess LLMs' fairness across various groups and developing more robust, fair, and generalizable detection frameworks.
Summary updated