TL;DR
This paper tackles the challenge of identifying "problematic" webpages (those promoting hate and violence) by developing a novel data collection and annotation strategy. They leverage existing hate speech datasets and GPT-4 to create a large-scale annotated dataset, which is then used to train a fine-tuned Transformer model that outperforms traditional hate speech detection methods.
What Problem Does This Paper Address?
The internet has seen a surge in user-generated content, including hateful and violent material. Problematic webpages can easily be indexed by search engines, appearing in search results even for innocuous queries, leading to a negative user experience and the spread of harmful content. While significant research exists on detecting hate speech in short-text formats (like social media posts), there's a notable gap in identifying and classifying entire webpages that promote hate and violence. This is a complex task due to the rich and varied features of webpages (URLs, titles, body text, links, ads) and the subtle, evolving nature of problematic content. Existing hate speech models often struggle with this due to their focus on shorter text and the difficulty of extracting meaningful features from entire webpages.
Methods
The authors propose a three-pronged approach:
- Webpage Data Curation: They start by leveraging existing hate speech short-text datasets. These datasets are pre-processed to remove noise and then used as queries for popular search engines (Google, Bing) to mine relevant webpages. This strategy aims to find webpages that are likely to be problematic when associated with hate speech keywords.
- GPT-4 Annotation: Due to the cost and complexity of manual annotation, they employ GPT-4 for labeling. They experiment with different prompting strategies, starting with a "Basic Prompt" and progressing to a "Precise Prompt" with a three-point scale (Problematic, Topically Sensitive, Clean) to better capture nuances. Finally, they use a "Few-shot Prompt" that includes summarized examples to guide GPT-4, achieving high-quality annotations.
- Model Training: A fine-tuned Transformer-based model (specifically Longformer, chosen for its ability to handle long sequences) is trained on the curated and annotated webpage data. They compare its performance against other models like BERT and HateBERT.
Datasets
- Source Hate Speech Datasets: The authors utilized several publicly available hate speech datasets to seed their webpage mining process. These include datasets from Davidson et al. (2017), de Gibert et al. (2018), Mollas et al. (2022), ElSherief et al. (2021), and Kennedy et al. (2020).
- Newly Introduced Benchmark: The paper introduces a novel, large-scale annotated dataset of problematic webpages. This dataset consists of approximately 21k problematic, 44k topically sensitive, and 42k clean webpages, totaling around 107k webpages. The "Topically Sensitive" category serves as valuable counterfactual data to improve model robustness.
Key Insights and Ideas
- Leveraging Search Engines for Data Mining: Using existing hate speech text as queries for search engines is an effective strategy to discover potentially problematic webpages, overcoming the challenge of directly mining such content.
- GPT-4 as a Powerful Annotator: GPT-4, with carefully crafted prompts (especially the few-shot approach with summarized examples), can achieve near human-level accuracy in annotating complex data like webpages, significantly reducing the cost and effort of manual annotation.
- The Importance of Nuance in Labeling: A simple binary classification (problematic/not problematic) is insufficient. Introducing a "Topically Sensitive" category helps to distinguish between content that discusses sensitive topics and content that actively promotes hate or violence, leading to more robust models.
- Webpage Features are Crucial: While body text is a primary source of information, URL and Title also provide significant context and are essential for accurate webpage classification. Combining these features yields the best results.
Key Results
- Annotation Performance: The "Precise Instructions with Few-shot Examples" prompt for GPT-4 achieved high-quality annotations, with inter-annotator agreement (measured by Cohen's kappa) of 0.87 when compared to human experts.
- Model Performance: The fine-tuned Longformer model achieved an overall F1-score of 87.6% for problematic webpage classification. This significantly outperformed traditional hate speech classification models trained solely on short-text data, showing a 13.7% gain in overall F1-score.
- Feature Importance: Combining URL, Title, and Body Text features yielded the highest F1-score (87.6%), demonstrating the collective importance of these elements for accurate classification.
Limitations and Future Work
- Language Limitation: The dataset and model are currently limited to the English language. The authors acknowledge that the model has not been evaluated in other languages.
- Future Work: The authors suggest extending the work by incorporating additional features like advertisements, link connections, and domain authority. They also propose creating multilingual datasets and models for problematic webpage classification.
Summary updated