Abstract
Detecting abusive language is a persistent challenge as it evolves in real-time, often shifting from explicit slurs to nuanced, implicit attacks. Traditional machine learning models are brittle when faced with new domains, such as COVID-related hate speech. This paper introduces a novel approach using Testing Concept Activation Vectors (TCAV) to quantify a model's sensitivity to human-defined concepts. We propose the Degree of Explicitness (DoE) metric to identify implicit abuse and guide data augmentation. Our experiments demonstrate that selecting implicit examples for retraining significantly improves model generalizability, outperforming standard uncertainty-based methods while preserving performance on original tasks.
Introduction
Content moderation systems frequently struggle with the "cat-and-mouse" nature of online abuse. When global events like the COVID-19 pandemic occur, new forms of hostility emerge rapidly. These often involve "implicit" abuse—subtle, context-dependent attacks that avoid traditional "red flag" keywords. While general-purpose classifiers can identify explicit insults in new domains, they often fail to detect these nuanced forms of hate.
The Challenge of Implicit Abuse
A significant gap exists in how models generalize across domains. Models trained on large, general datasets like Wikipedia Toxicity or Founta (Twitter) perform poorly when tested on datasets with high proportions of implicit abuse, such as the East-Asian Prejudice (EA) dataset.
As shown in the table above, models trained on pre-COVID data (Wiki, Founta) achieve dismal F1-scores (0.27–0.29) on the implicit-heavy EA dataset, despite performing better on the more explicit COVID-HATE (CH) dataset. This failure is not merely a vocabulary issue; while 32% of the words in COVID datasets are pandemic-specific, the primary challenge is the weaponization of neutral terms in implicit contexts.
Methodology: TCAV and Degree of Explicitness
To address this, we adapt Testing Concept Activation Vectors (TCAV)—originally a computer vision interpretability technique—to NLP. The process involves:
- Concept Definition: Users provide a small set (~100) of examples representing a concept, such as "implicit anti-Asian abuse."
- Vector Space Mapping: Using a RoBERTa-based classifier, we calculate a Concept Activation Vector (CAV) based on the model's internal activations for these examples.
- Sensitivity Testing: Directional derivatives measure how much the model’s prediction changes when the input representation is nudged toward the concept vector.
- Degree of Explicitness (DoE): We derive a per-instance metric that measures how much an unlabeled sentence aligns with the "explicit abuse" concept.
By using DoE, we can identify samples that are highly implicit (low DoE). We hypothesize that these samples are the most informative for the model to learn during data augmentation.
Results and Analysis
We compared DoE-guided data augmentation against standard confidence-based (uncertainty) sampling. The goal was to update a general classifier using a small amount of data from the target domain.
The results indicate that DoE-based sampling is consistently superior for the EA dataset. Selecting 3,000 implicit examples for augmentation boosted the F1-score on the EA dataset to 0.61, compared to significantly lower performance for confidence-based sampling. Furthermore, the DoE method proved more stable; it maintained performance on the original Wikipedia test set within 2% of the baseline, whereas confidence-based sampling caused drops of up to 6%.
Conclusion
Interpretability tools like TCAV can be repurposed from mere explanation to active model improvement. By quantifying the "Degree of Explicitness," we can strategically select training data that targets a model's specific blind spots—namely, implicit abuse. This approach offers a more efficient and stable path for adapting content moderation systems to rapidly evolving online discourse.
Summary updated