TL;DR
This paper tackles the challenge of moving beyond simple hate speech classification to a more nuanced understanding by identifying who is being targeted and what specific characteristics are being attributed to them. They propose a novel approach that adapts a generic concept extraction model to the domain of social media hate speech, achieving promising results in identifying these targets and aspects.
What Problem Does This Paper Address?
Current research on hate speech primarily focuses on classifying entire posts into broad categories like "hate speech," "racism," or "sexism." While useful for detection and removal, this approach is insufficient for combating hate speech effectively. Laws in many countries protect hate speech, making outright removal impossible, and studies suggest that education and counter-narratives are crucial. To enable these, a deeper analysis is needed: identifying the specific entities (targets) being attacked and the particular attributes or characteristics (aspects) being ascribed to them. For example, simply knowing a post is "racist" isn't as informative as knowing it targets "Muslims of Palestine" and claims they are "terrorists." This paper aims to fill this gap by developing a method for extracting these fine-grained targets and aspects.
Methods
The core of the proposed method involves adapting a state-of-the-art open-domain concept extraction model, built on pointer-generator networks, to the specific domain of social media hate speech. This generic model is first trained on diverse datasets. The key innovation lies in the domain adaptation stage. This involves:
- Target and Aspect Definitions: Clearly defining "target" as the entity in focus and "aspect" as a characteristic attributed to the target, often with a pejorative connotation.
- Concept Extraction: Using the generic model to extract potential target and aspect candidates from hate speech posts.
- Domain-Specific Weighting: Developing algorithms (Algorithms 1, 2, and 3) that fine-tune weight variables to prioritize certain types of candidates (e.g., nominal vs. adjectival targets, different aspect types) based on their relevance in the hate speech context. This includes considering subword information and ranking candidates based on frequency and other linguistic features.
- Iterative Refinement: The process involves iterative application of algorithms to refine target and aspect identification, aiming to capture nuances like compound targets and non-nominal aspects.
Datasets
The primary dataset used is the 'sexism' and 'racism' partitions of the Waseem and Hovy (2016a) dataset. This dataset contains 5,355 positive instances of hate speech. It was split into disjoint reference (90%), development (2%), and test (8%) sets. The reference set was used for identifying domain-specific nominal group target candidates, and the development set was used for fine-tuning the discrete variables in the algorithms. An additional dataset from Dong et al. (2014) (negative sentiment subset) was used to verify the dataset-specificity of the fine-tuning procedure.
Key Insights and Ideas
- Beyond Classification: The paper strongly advocates for moving beyond simple hate speech classification to a more granular analysis of targets and aspects for effective counter-strategies.
- Open-Class Targets and Aspects: The approach aims to identify "open-class" targets and aspects, meaning it doesn't rely on a predefined, limited set of categories, allowing for the discovery of new or intersecting forms of hate.
- Importance of Domain Adaptation: A generic concept extraction model, while powerful, requires significant adaptation to perform well on the specific linguistic characteristics and idiosyncrasies of hate speech data.
- Subword Information is Crucial: Recognizing that targets can be part of compound words (e.g., "Daeshbags") necessitates considering subword units for more comprehensive extraction.
- Aspects Can Be Diverse: Aspects are not limited to simple adjectives; they can be verbal groups or even nominal concepts themselves, adding complexity to their identification.
Key Results
The proposed GetTA_Pair model achieved promising results:
- Target Identification: Achieved an accuracy of 0.65 for exact match on the development set and 0.57 for exact match on the test set.
- Aspect Identification: Performance for aspect recognition was generally lower but still showed potential, with a partial aspect match accuracy of 0.74.
- Comparison to BERT: A BERT model fine-tuned on the development set showed lower exact match accuracy for targets compared to GetTA_Pair, though it performed better on partial matches, suggesting BERT might struggle with the specific composition of aspects in this domain.
- Variable Importance: The study highlighted the importance of different weight variables, with nominal target candidates and proper names being prioritized for target detection, and specific ordering for aspect detection.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Aspect Identification Quality: The quality of aspect identification needs further improvement.
- Language Independence: The current implementation is somewhat language-specific; future work aims to use distance learning for language independence.
- Generalizability: Further experiments on diverse hate speech datasets are needed to demonstrate the model's generalizability.
- Multiple Targets: The current model primarily focuses on single targets per post; extending it to capture multiple targets would be beneficial.
- Transformer Models: The paper suggests that pre-training transformer models like BERT on concept-annotated datasets could potentially improve exact match figures for target identification.
Summary updated