Parallel Corpora for bi-Directional Statistical Machine Translation for Seven Ethiopian Language Pairs

Solomon Teferra Abate, Michael Melese, Martha Yifiru Tachbelie, Million Meshesha, Solomon Atinafu, Wondwossen Mulugeta, Yaregal Assabie, Hafte Abera, Binyam Ephrem, Tewodros Abebe, Wondimagegnhue Tsegaye, Amanuel Lemma, Tsegaye Andargie, Seifedin ShifawCOLING 2018

TL;DR

This paper tackles the challenge of building machine translation systems for under-resourced Ethiopian languages by creating and evaluating parallel corpora. The authors demonstrate that while these corpora enable basic bi-directional statistical machine translation (SMT), the morphological complexity of Ethio-Semitic languages significantly hinders performance, especially when they are the target language.

What Problem Does This Paper Address?

Natural Language Processing (NLP) applications, including machine translation (MT), are increasingly vital for bridging communication gaps. However, for many languages, particularly those in developing regions like Ethiopia, the lack of readily available linguistic resources, especially parallel corpora, is a major bottleneck. These corpora are essential for training statistical machine translation (SMT) models. Without them, researchers are forced to create small, custom datasets, which limits the scope and comparability of their experiments. This paper addresses this by developing and releasing parallel corpora for seven Ethiopian language pairs and evaluating their utility through initial SMT experiments.

Methods

The core of this work involves the creation of parallel corpora and their subsequent use in SMT experiments.

  • Corpus Creation: The researchers collected parallel texts primarily from religious domains, specifically the Holy Bible and related documents from sources like Jehovah's Witnesses, Ethiopicbible, Ebible, and Geez experience. A web crawler was employed to extract texts, and Python libraries (requests, BeautifulSoup) were used for parsing and text extraction. URLs were structured to capture book names, chapters, and verses for each language.
  • Pre-processing: A crucial step involved automatic text pre-processing to handle the diverse formats of the collected data. This included:
  • Character Normalization: Addressing variations in Amharic characters that represent similar sounds or have redundant roles, consolidating them into a single, frequently used character to avoid treating semantically identical words as different.
  • Sentence Tokenization and Alignment: Tokenizing lines containing multiple sentences and establishing alignment criteria. A key challenge was aligning single verses in Ge'ez with potentially multiple verses on a single line in other languages, requiring merging of Ge'ez verses.
  • SMT Experiments: Bi-directional SMT experiments were conducted using the Moses SMT toolkit. Giza++ was used for word and phrase alignment, and the SRILM toolkit for language model development. The Bilingual Evaluation Under Study (BLEU) score was used for automatic evaluation.

Datasets

The paper introduces newly created parallel corpora for seven Ethiopian language pairs:

  • Amharic - Tigrigna
  • Amharic - Afan Oromo
  • Tigrigna - Afan Oromo
  • Amharic - Wolaytta
  • Ge'ez - Amharic
  • Wolaytta - Afan Oromo
  • Tigrigna - Wolaytta

These corpora were derived from religious texts and are characterized by varying sizes, as detailed in Table 1. The data was split into 80% for training, 10% for tuning, and 10% for testing for the SMT experiments.

Key Insights and Ideas

  • Morphological Complexity is a Major Hurdle: The most significant finding is that the inherent morphological complexity of Ethio-Semitic languages (Amharic, Tigrigna, Ge'ez) poses a substantial challenge for SMT performance, particularly when these languages are the target. This complexity leads to a higher number of tokens and word types even for the same number of sentences, exacerbating data sparsity.
  • Directionality Matters: Translation performance varies significantly depending on the direction of translation. Systems translating from Ethio-Semitic languages generally perform better than those translating into them. This is attributed to one-to-many alignment issues when translating into morphologically rich languages, whereas many-to-one alignment is more favorable when translating from them.
  • Data Size vs. Morphology: While larger datasets generally lead to better performance, the paper suggests that morphological complexity can have a more pronounced negative impact than data size alone.
  • Syntactic Differences Also Play a Role: The paper notes that syntactic differences, such as word order (SOV vs. VSO), can also affect SMT performance, though the impact of morphology appears more dominant.

Key Results

  • The developed parallel corpora were found to be usable for initial bi-directional SMT experiments, demonstrating that the corpora can support further research.
  • BLEU scores varied significantly across language pairs and directions. For instance, Amharic-Tigrigna achieved a BLEU score of 21.22 in one direction, while Tigrigna-Wolaytta achieved only 2.2 in the other.
  • Generally, translation systems from Ethio-Semitic languages to Cushitic or Omotic languages showed better BLEU scores than the reverse direction.
  • Amharic was identified as the most morphologically complex language within the Ethio-Semitic family based on token and word type counts.

Limitations and Future Work

  • Data Size: The authors acknowledge that the size of the collected corpora, while usable, is still relatively small, which limits the conclusions that can be drawn.
  • Domain Specificity: The corpora are limited to the religious domain, which may not be representative of general language use.
  • Future Work:
    • Handling Morphological Complexity: The primary future direction is to develop methods to mitigate the negative impact of morphological complexity. This includes exploring morphological segmentation (using morphemes instead of words as units) as suggested by previous work.
  • ANN Modeling: The authors plan to investigate the use of Artificial Neural Network (ANN) modeling, which is the current trend in MT, but acknowledge that this will require significantly more data.
  • Expanding Domains: Efforts will be made to develop and organize parallel corpora for Ethiopian languages across different domains to improve the generalizability of MT systems.

Summary updated

BibTeX
@inproceedings{abate2018parallel,
  author = {Solomon Teferra Abate and Michael Melese and Martha Yifiru Tachbelie and Million Meshesha and Solomon Atinafu and Wondwossen Mulugeta and Yaregal Assabie and Hafte Abera and Binyam Ephrem and Tewodros Abebe and Wondimagegnhue Tsegaye and Amanuel Lemma and Tsegaye Andargie and Seifedin Shifaw},
  title = {{Parallel Corpora for bi-Directional Statistical Machine Translation for Seven Ethiopian Language Pairs}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure