A Unified Model for Arabizi Detection and Transliteration using Sequence-to-Sequence Models

Ali Shazal, Aiza Usman, Nizar HabashWANLP @ COLING 2020

Abstract

Arabizi—the practice of writing Arabic dialects using Roman characters and numerals—is a prevalent form of communication on social media and chat platforms. However, its lack of standardization, frequent code-switching with languages like English, and complex word-boundary mismatches pose significant hurdles for conventional Natural Language Processing (NLP) systems. This paper introduces a unified character-level sequence-to-sequence (Seq2Seq) model designed to detect and transliterate Arabizi into Conventional Orthography for Dialectal Arabic (CODA) in a single step. By leveraging hybrid embeddings and a sliding window context, the proposed model achieves a word accuracy of 80.6%, significantly surpassing traditional multi-stage pipeline baselines.

Introduction: The Arabizi Challenge

Arabizi is a spontaneous, non-standardized orthography where users substitute Arabic letters with Roman characters and numbers based on phonetic similarity (e.g., using '3' for the letter 'ع'). This medium presents three primary challenges for NLP:

  1. Lack of Standardization: A single word can have dozens of spelling variations; for instance, the word habibi ("my beloved") appears in the researchers' dataset with 24 different spellings (e.g., 7abiby, hbebe).
  2. Code-Switching: Sentences frequently mix Arabic with foreign languages (English or French) and emojis within the same string.
  3. Orthographic Mismatches: Word boundaries in Arabizi do not always align with standard Arabic script. A single Arabizi word might need to be split into two in Arabic, or conversely, two Arabizi words might need to be merged.

Previously, researchers handled these issues using a multi-stage pipeline: first "detecting" the language of each word and then "transliterating" the Arabic portions. This paper proposes a unified model that handles both tasks simultaneously to avoid error propagation.

Methodology: A Unified Seq2Seq Approach

The core of the system is a character-level Seq2Seq model utilizing Gated Recurrent Units (GRUs) and an attention mechanism. To address the unique complexities of Arabizi, several technical innovations were implemented:

  • Hybrid Embeddings: The model combines character-level embeddings with FastText word embeddings. This dual-input strategy allows the network to capture both fine-grained spelling variations and broader semantic context.
  • The "Pass-Through" Mechanism: To handle code-switching, the model is trained to map foreign words and emojis to a special placeholder symbol (#) during the machine learning phase. These are restored to their original Roman-script form during post-processing.
  • Alignment Tokens: Special markers ([+] for merges and [-] for splits) are used to train the model on the word-boundary adjustments required for CODA.
  • Sliding Window Context: To prevent the "forgetfulness" common in long-sequence RNNs, the researchers adopted a WORD2WORD +/- 1 approach. The model processes one word at a time while maintaining visibility of the immediate preceding and following words for context.
The Seq2Seq model architecture featuring BiGRU and GRU layers with character and word embeddings.

Experimental Setup and Results

The model was trained and evaluated using the ARABIZICORPUS, specifically the BOLT Egyptian Arabic SMS/Chat dataset, comprising over 287,000 words. This dataset includes manually validated transliterations and tags for various token types. Additionally, 1 million words of unannotated Arabizi were used to pre-train the FastText embeddings.

The unified Seq2Seq model was benchmarked against a Maximum Likelihood Estimate (MLE) baseline. The results demonstrate the superiority of the Seq2Seq approach, particularly when incorporating local context.

Comparative performance metrics (Accuracy and BLEU) for MLE and Seq2Seq models across different context settings.

As shown in the results, the WORD2WORD +/- 1 configuration achieved the highest performance, with a word accuracy of 80.6% (a 9.9% absolute improvement over the baseline) and a BLEU score of 58.7. A key finding was that "Line-to-Line" (sentence-level) models performed worse than windowed models, suggesting that local context is more critical for transliteration than global sentence structure. Furthermore, the Seq2Seq model proved robust in handling out-of-vocabulary (OOV) words, whereas the baseline failed on any unseen spelling variations.

Conclusion and Future Work

This research demonstrates that a unified Seq2Seq model can effectively bridge the gap between informal Arabizi and standardized Arabic script. By integrating language detection into the transliteration process, the model reduces complexity and improves accuracy.

However, limitations remain. The current model is optimized for the Egyptian dialect and may require retraining for others, such as Moroccan or Levantine. Additionally, the computational cost of Seq2Seq inference is significantly higher than simple lookup baselines. Future work will focus on expanding dialectal coverage and exploring hybrid systems that combine the speed of baselines with the robustness of neural models for unseen data.

Summary updated

BibTeX
@inproceedings{shazal2020unified,
  author = {Ali Shazal and Aiza Usman and Nizar Habash},
  title = {{A Unified Model for Arabizi Detection and Transliteration using Sequence-to-Sequence Models}},
  booktitle = {WANLP @ COLING},
  year = {2020},
}
Figure