EkoHate: Abusive Language and Hate Speech Detection for Code-switched Political Discussions on Nigerian Twitter

Comfort Eseohen Ilevbare, Jesujoba O. Alabi, David Ifeoluwa Adelani, Firdous Damilola Bakare, Oluwatoyin Bunmi Abiola, Oluwaseyi A. Adeyemo2024 · DOI

Abstract

Nigerians have a notable online presence and actively discuss political and topical matters. This was particularly evident throughout the 2023 general election, where Twitter was used for campaigning, fact-checking and verification, and even positive and negative discourse. However, little or none has been done in the detection of abusive language and hate speech in Nigeria. In this paper, we curated codeswitched Twitter data directed at three musketeers of the governorship election on the most populous and economically vibrant state in Nigeria; Lagos state, with the view to detect offensive speech in political discussions. We developed EKOHATE—an abusive language and hate speech dataset for political discussions between the three candidates and their followers using a binary (normal vs offensive) and finegrained four-label annotation scheme. We analysed our dataset and provided an empirical evaluation of state-of-the-art methods across both supervised and cross-lingual transfer learning settings. In the supervised setting, our evaluation results in both binary and four-label annotation schemes show that we can achieve 95.1 and 70.3 F1 points respectively. Furthermore, we show that our dataset adequately transfers very well to three publicly available offensive datasets (OLID, HateUS2020, and FountaHate), generalizing to political discussions in other regions like the US.

Introduction

Most hate speech detection research is centered on Western contexts, particularly US elections. However, online discourse in Nigeria is linguistically complex and culturally specific. During the 2023 general elections, Twitter became a primary battleground for campaigning and discourse, yet existing models often fail to grasp the nuances of Nigerian "code-switching"—where users fluidly move between English, Yoruba, and Nigerian Pidgin (Naija).

The authors identify a "representation gap": standard datasets don't account for the specific slurs, sarcasm, or "contemptuous" tones used in Nigerian politics. Without localized data, automated moderation tools remain ineffective in one of Africa's most active digital democracies.

The EKOHATE Dataset

The researchers manually extracted 3,398 tweets and replies directed at the three leading candidates for the Lagos State governorship. The dataset is multilingual, featuring English (63.2%), Code-switched text (26%), Nigerian Pidgin (7.3%), and Yoruba (3.5%).

Distribution of tweet categories across different languages in the EKOHATE dataset.

Beyond simple binary labels, the authors implemented a four-label scheme: Normal, Abusive, Hateful, and a novel Contempt category. The "contempt" label captures tweets that express strong dislike in a neutral tone without overt slurs (e.g., "You will still be voted out of office sir").

Methodology

The study fine-tuned Twitter-RoBERTa-base, a language model pre-trained on social media text. To improve performance on difficult multi-class tasks, the researchers employed ensembling via majority voting across multiple model runs.

To test robustness, they performed cross-corpus transfer learning, training models on EKOHATE and testing them on established US-centric datasets (OLID, HateUS2020, and FountaHate) and vice versa.

Experimental Results

The evaluation focused on both binary and fine-grained classification schemas. As shown in the results table below, the model performs exceptionally well on binary tasks but faces challenges as the granularity increases.

Performance (F1 scores) of the EKOHATE dataset across different annotation schemas.

Key Findings

  • Binary vs. Fine-grained: The model achieved a high 95.1 F1 score for binary "Normal vs. Offensive" detection, but this dropped to 70.3 F1 for the four-label task.
  • The Difficulty of Hate and Contempt: The "Hateful" class was frequently misclassified as "abusive," while "contempt" proved difficult for models to distinguish from "normal" speech.
  • Generalization Power: Models trained on EKOHATE transferred surprisingly well to US-based datasets, suggesting the Nigerian data captures a broad spectrum of offensive language.
  • Linguistic Challenges: Performance was notably lower on Nigerian Pidgin (Naija) compared to English or Yoruba, likely due to non-standardized spelling and its low-resource nature.

Conclusion and Future Work

While EKOHATE provides a strong foundation for localized hate speech detection, challenges remain regarding data imbalance in the "Hateful" and "Contempt" classes. Future work should focus on expanding the representation of Yoruba and Pidgin and exploring few-shot learning techniques to better identify rare instances of high-impact hate speech.

Summary updated

Abstract

Nigerians have a notable online presence and actively discuss political and topical matters.This was particularly evident throughout the 2023 general election, where Twitter was used for campaigning, fact-checking and verification, and even positive and negative discourse.However, little or none has been done in the detection of abusive language and hate speech in Nigeria.In this paper, we curated codeswitched Twitter data directed at three musketeers of the governorship election on the most populous and economically vibrant state in Nigeria; Lagos state, with the view to detect offensive speech in political discussions.We developed EKOHATE-an abusive language and hate speech dataset for political discussions between the three candidates and their followers using a binary (normal vs offensive) and finegrained four-label annotation scheme.We analysed our dataset and provided an empirical evaluation of state-of-the-art methods across both supervised and cross-lingual transfer learning settings.In the supervised setting, our evaluation results in both binary and four-label annotation schemes show that we can achieve 95.1 and 70.3 F1 points respectively.Furthermore, we show that our dataset adequately transfers very well to three publicly available offensive datasets (OLID, HateUS2020, and FountaHate), generalizing to political discussions in other regions like the US.
BibTeX
@misc{ilevbare2024ekohate,
  author = {Comfort Eseohen Ilevbare and Jesujoba O. Alabi and David Ifeoluwa Adelani and Firdous Damilola Bakare and Oluwatoyin Bunmi Abiola and Oluwaseyi A. Adeyemo},
  title = {{EkoHate: Abusive Language and Hate Speech Detection for Code-switched Political Discussions on Nigerian Twitter}},
  year = {2024},
  doi = {10.18653/v1/2024.woah-1.3},
}
Figure