Morphotactic Modeling in an Open-source Multi-dialectal Arabic Morphological Analyzer and Generator

Nizar Habash, Reham Marzouk, Salam Khalifa, Christian KhairallahOpenReview 2022 · OpenReview

Abstract

Arabic is a morphologically rich and complex language, with numerous dialectal variants. Previous efforts on Arabic morphology modeling focused on specific variants and specific domains using a range of techniques with different degrees of linguistic modeling transparency. In this paper we propose a new approach to modeling Arabic morphology with an eye towards multi-dialectness, resource openness, and easy extensibility and use. We demonstrate our approach by modeling verbs from Standard Arabic and Egyptian Arabic, within a common framework, and with high coverage.

Introduction

Arabic morphology presents a significant challenge for Natural Language Processing (NLP) due to its templatic nature and the proliferation of distinct dialects. A single verb root can generate thousands of surface forms through the interleaving of roots and patterns, combined with various prefixes, suffixes, and clitics. Existing tools often struggle to balance linguistic accuracy with computational efficiency, frequently focusing on Modern Standard Arabic (MSA) while neglecting dialects like Egyptian Arabic (EGY). There is a critical need for a framework that is both linguistically principled and computationally efficient.

The CAMELMORPH Framework

To address these challenges, we introduce CAMELMORPH, an open-source framework designed for multi-dialectal morphological modeling. CAMELMORPH bridges the gap between abstract linguistic models and shallow tabulated solutions by utilizing a pipeline that transforms human-readable specifications into high-performance engines.

The CAMELMORPH pipeline architecture, from linguistic specifications to morphological engines.

The framework relies on four primary specification tables: Order, which defines the sequence of morphemes; Lexicon, which lists base stems; Morph, which defines allomorphs and their selection conditions; and Regex, which handles orthographic adjustments. These specifications are compiled into a database that ensures morphotactic validity, which is then integrated into the Camel Tools library for real-time analysis and generation.

Innovative Modeling Techniques

A key innovation in CAMELMORPH is the "Stem Buffer" system. Rather than expanding the lexicon to account for every minor stem variation, the system uses small buffer fragments to handle complex interactions, such as vowel changes triggered by specific suffixes. This approach maintains a compact lexicon while ensuring high linguistic fidelity.

Furthermore, the system employs condition-based selection instead of complex rewrite rules. By using "Required Conditions" (e.g., v-suff for vowel-initial suffixes), the logic remains transparent and easily auditable by linguists. This shared framework allows for consistent modeling across dialects, paving the way for future research into code-switching between MSA and EGY.

Sample morphological specifications illustrating the use of stem buffers and condition-based selection for MSA perfective verbs.

Evaluation and Results

The framework was evaluated using gold-standard datasets: the Penn Arabic Treebank (PATB) for MSA and the LDC Egyptian Arabic Treebank (ARZATB) for EGY.

The MSA model achieved a recall of 98.4% on verbal analyses in the PATB and produced 2.9 times more analyses than previous standards, effectively filling gaps in imperative and passive forms. For EGY, the model reached 69.9% recall. While lower, this reflects the lack of standardized orthography in dialectal Arabic; many discrepancies were due to inconsistencies in the reference data rather than model failure.

Conclusion

CAMELMORPH provides a transparent, extensible, and high-coverage framework for Arabic morphology. By sharing a common morpheme order framework between MSA and EGY, it facilitates the development of multi-dialectal tools. Future work will expand the system to include nouns and adjectives and improve robustness against non-standard orthography. The framework is publicly available at morph.camel-lab.com.

Summary updated

Abstract

Arabic is a morphologically rich and complex language, with numerous dialectal variants. Previous efforts on Arabic morphology modeling focused on specific variants and specific domains using a range of techniques with different degrees of linguistic modeling transparency. In this paper we propose a new approach to modeling Arabic morphology with an eye towards multi-dialectness, resource openness, and easy extensibility and use. We demonstrate our approach by modeling verbs from Standard Arabic and Egyptian Arabic, within a common framework, and with high coverage.
BibTeX
@misc{habash2022morphotactic,
  author = {Nizar Habash and Reham Marzouk and Salam Khalifa and Christian Khairallah},
  title = {{Morphotactic Modeling in an Open-source Multi-dialectal Arabic Morphological Analyzer and Generator}},
  year = {2022},
  url = {https://openreview.net/forum?id=qtlEEm3wZz},
  howpublished = {OpenReview},
}
Figure