Modeling Profanity and Hate Speech in Social Media with Semantic Subspaces

Vanessa Hahn, Dana Ruiter, Thomas Kleinbauer, Dietrich Klakow

Abstract

Automated moderation of online hate speech and profanity is hindered by severe data sparsity, particularly for non-English languages. This research proposes a method to isolate a "profane semantic subspace" within language model embeddings using a small set of "minimal pairs"—profane words or sentences and their neutral counterparts. By applying Principal Component Analysis (PCA) to these pairs, the authors demonstrate significant improvements in zero-shot classification across German, English, French, and Arabic. The approach proves highly efficient, requiring as few as 30–50 training pairs to achieve robust cross-lingual transfer, suggesting that this subspace captures a broad offensive semantic concept that generalizes across tasks and languages with minimal supervision.

Introduction

Detecting abusive language online is a critical yet challenging task due to the high subjectivity of labels and the scarcity of high-quality datasets outside of English. Standard pre-trained models like BERT often struggle in low-resource settings where annotation protocols vary. This paper investigates whether the mathematical representation of "profanity" can be isolated in an embedding space and leveraged for zero-shot detection across diverse languages.

Methodology

The core of the approach involves learning semantic subspaces at both word and sentence levels to model offensive language.

Minimal Pairs and Subspace Extraction

The authors utilize "minimal pairs": pairs of words or sentences identical in meaning except for the presence of profanity. For example, the German word pair Arschloch (asshole) and Mann (man) serves as a training point to define the profanity dimension.

These pairs are encoded using FastText or BERT. Principal Component Analysis (PCA) is then applied to the differences between paired embeddings to isolate the principal components (PCs) representing the "profanity" dimension. The study evaluates three representation types: raw embeddings (BASE), projections onto the raw profane subspace (PCA-RAW), and projections onto a mean-shifted normalized subspace (PCA-NORM).

Evaluation Framework

A simple Linear Discriminant Analysis (LDA) classifier is trained solely on the German subspace representations and tested on unseen benchmarks: GermEval-2019 (German), HASOC 2019 (English), and datasets for French and Arabic.

Results and Discussion

The subspace-based approach yielded massive improvements in zero-shot classification performance across all tested scenarios.

Cross-Lingual and Cross-Task Transfer

Subspace representations (PCA-RAW/NORM) significantly outperformed the standard BERT baseline, with F1 score gains ranging from +10.9 to +42.9. Notably, a subspace learned on German profanity successfully transferred to English (+20.6 F1) and Arabic (+22.4 F1). Furthermore, the model generalized from profanity detection to the more complex task of hate speech detection, indicating it captures a broad offensive semantic concept.

Low-Resource Efficiency

A key finding is the extreme data efficiency of the method. Performance typically plateaus with only 30 to 50 minimal pairs, making it highly suitable for low-resource languages where large annotated datasets are unavailable.

Insights on Normalization

Contrary to prior literature on bias mitigation, the authors found that mean-shift normalization is not strictly necessary; PCA-RAW often performed comparably to PCA-NORM. This suggests that perfectly aligned minimal pairs may not be required for identifying semantic directions in the future, potentially simplifying data collection.

Conclusion and Future Work

While the model shows a tendency toward "over-blacklisting" neutral tweets containing negative keywords (e.g., "crime"), its zero-shot capabilities are remarkably robust. Future research will explore building these subspaces using unaligned offensive and neutral corpora, further reducing the data collection burden for automated moderation systems.

Summary updated

BibTeX
@inproceedings{hahn2021modeling,
  author = {Vanessa Hahn and Dana Ruiter and Thomas Kleinbauer and Dietrich Klakow},
  title = {{Modeling Profanity and Hate Speech in Social Media with Semantic Subspaces}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure