Identifying Nuanced Dialect for Arabic Tweets with Deep Learning and Reverse Translation Corpus Extension System

Rawan Tahssin, Youssef Kishk, Marwan TorkiWANLP @ COLING 2020

Abstract

Identifying the country of origin for Arabic tweets is a complex task due to high linguistic similarity between dialects and significant data imbalance. This paper introduces the Reverse Translation Corpus Extension System (RTCES), a novel data augmentation strategy that leverages back-translation to highlight dialect-specific markers. By combining RTCES with a fine-tuned AraBERT model, the authors demonstrate superior performance over traditional machine learning and standard deep learning approaches in the NADI Shared Task.

Introduction

Arabic is a pluricentric language where Modern Standard Arabic (MSA) serves formal purposes, while regional dialects dominate daily communication and social media. Automatically identifying these dialects is essential for applications ranging from sentiment analysis to personalized search. However, the task is hindered by two primary challenges: the subtle linguistic nuances shared across dialects and the extreme imbalance in available training data, which often favors countries like Egypt and Iraq while neglecting others like Djibouti or Mauritania.

Distribution of training samples across 21 Arab countries, highlighting the significant data imbalance.

Methodology

The core contribution of this work is the Reverse Translation Corpus Extension System (RTCES). This system addresses data scarcity in underrepresented dialects through a specialized back-translation process.

Reverse Translation Corpus Extension System (RTCES)

Unlike standard back-translation, which aims for semantic parity, RTCES focuses on the lexical difference between original and translated texts. The process involves translating Arabic tweets into English and then back into Arabic. Because translation engines typically normalize dialectal Arabic into MSA, the "delta" or difference between the original and the back-translated text often isolates the most distinctive dialectic markers. These extracted differences are used to create new, synthetic training sentences.

The RTCES workflow: from pre-processing to extracting the lexical difference between original and back-translated Arabic.

Model Architectures

The researchers evaluated several models to determine the most effective approach for 21-way classification:

  • TF-IDF + Logistic Regression: A baseline for keyword-based classification.
  • fastText + Logistic Regression/Bi-LSTM: Utilizing sub-word embeddings to capture morphological nuances.
  • AraBERT: A Transformer-based model pre-trained on a massive 3-billion-word Arabic corpus. This model was fine-tuned specifically for the dialect identification task.

Experimental Setup and Results

The study utilized the NADI Shared Task 2020 dataset, consisting of 21,000 training tweets. By applying RTCES, the authors expanded the training set to 32,417 sentences, specifically bolstering the representation of minority classes.

The fine-tuned AraBERT model emerged as the top performer, achieving a Macro Average F1 score of 20.34% on the test set. This result secured the team (CodeLyoko) the 7th position out of 18 participants. Notably, the close alignment between development (20.73%) and test (20.34%) scores suggests that the RTCES augmentation successfully improved model generalization without overfitting.

Discussion and Future Work

The experiments revealed that while simple TF-IDF models are competitive, the combination of massive pre-training (AraBERT) and targeted data augmentation (RTCES) provides the necessary edge for nuanced dialect detection. However, an F1 score of ~20% underscores the inherent difficulty of country-level classification.

Future research will focus on refining hierarchical models (predicting regions before countries) and integrating Named Entity Recognition (NER). Identifying local landmarks, public figures, or regional foods could serve as high-confidence signals to further distinguish between linguistically similar dialects.

Summary updated

BibTeX
@inproceedings{tahssin2020identifying,
  author = {Rawan Tahssin and Youssef Kishk and Marwan Torki},
  title = {{Identifying Nuanced Dialect for Arabic Tweets with Deep Learning and Reverse Translation Corpus Extension System}},
  booktitle = {WANLP @ COLING},
  year = {2020},
}
Figure