Unlock Tigrigna NLP: Design and Development of Morphological Analyzer for Tigrigna Verbs Using Hybrid Approach

Hagos Gebremedhin Gebremeskel, Chong Feng, Heyan HuangSSRN 2023 · DOI

Abstract

Tigrigna, a Semitic language spoken by over seven million people in Ethiopia and Eritrea, presents significant challenges for Natural Language Processing (NLP) due to its complex, non-concatenative morphology. This paper details the development of a high-performance morphological analyzer for Tigrigna verbs. By employing a hybrid approach that integrates Memory-Based Learning (MBL) with linguistic rule-based refinement, the system achieves an accuracy of 95.6%. This tool serves as a critical foundation for advanced Tigrigna NLP applications, including machine translation and information retrieval.

Introduction

Tigrigna is characterized as a "morphologically rich" language. Unlike English, where verb variations are limited, a single Tigrigna verb root can generate hundreds of forms through the application of prefixes, suffixes, and internal vowel changes. For computational systems, these variations (e.g., "በለዐ" vs. "ምስበለዐ") appear as distinct data strings, hindering effective processing. A morphological analyzer is essential to decompose these inflected forms into their constituent morphemes and identify the core root. While such tools exist for related languages like Amharic, Tigrigna's unique complexity necessitated a specialized, comprehensive solution.

Methodology

The researchers implemented a hybrid architecture to leverage the strengths of both statistical learning and linguistic expertise.

Memory-Based Learning (MBL)

The system utilizes the TiMBL framework, specifically the IB2 and TRIBL2 algorithms. MBL functions by analogy; it stores instances of inflected verbs and classifies new, unseen words by comparing them to the most similar examples in its memory. To provide necessary context for these algorithms, a "windowing" feature extraction method was used, where each character is analyzed alongside its 10 preceding and 10 succeeding characters.

Rule-Based Refinement

To address "edge cases" where machine learning often falters—such as irregular verbs and copulas—the authors integrated Finite State Transducer (FST) rules. These linguistic rules provide a secondary layer of precision, refining the output of the MBL component.

Architecture of the hybrid morphological analyzer for Tigrigna verbs.

Morpheme Annotation

Verbs were manually annotated into a structured segment: NegPref/PosPre/SMS/OMS + Stem + SufCirc/NegSuf. This structured data allowed the model to learn the specific morphological roles of various segments within a word.

Data and Evaluation

Due to the lack of pre-existing resources for Tigrigna, a custom dataset was developed:

  • Core Dataset: 589 manually annotated stem verbs.
  • Expanded Dataset: 8,112 instances of inflected, derived, and compound verbs extracted from the core stems.
  • Evaluation: The model was evaluated using 10-fold cross-validation, ensuring robust performance across different subsets of the data.

Results

The experimental results demonstrated the efficacy of the hybrid approach:

  • IB2 Algorithm: Achieved 93.24% accuracy with optimized parameters and showed superior data compression.
  • TRIBL2 Algorithm: Achieved 92.31% accuracy.
  • Hybrid System: The integration of rule-based refinement for irregulars pushed the final performance to 95.6%.

The study highlighted a trade-off between speed and precision; while IB2 was more accurate and memory-efficient, it was slower than TRIBL2. Furthermore, the success of the 21-character window confirmed that Tigrigna's morphological markers are distributed throughout the word, requiring a broad contextual view for accurate analysis.

Conclusion

The development of this hybrid morphological analyzer represents a significant step forward for Tigrigna NLP. By combining the "brute force" of memory-based learning with the precision of linguistic rules, the system achieves high accuracy despite the language's inherent complexity and limited digital resources. Future work will focus on expanding the analyzer's scope to include nouns and adjectives and exploring more modern architectures to further enhance performance.

Summary updated

BibTeX
@misc{gebremeskel2023unlock,
  author = {Hagos Gebremedhin Gebremeskel and Chong Feng and Heyan Huang},
  title = {{Unlock Tigrigna NLP: Design and Development of Morphological Analyzer for Tigrigna Verbs Using Hybrid Approach}},
  year = {2023},
  doi = {10.2139/ssrn.4627238},
  howpublished = {SSRN},
}
Figure