Deep Learning Meets Egyptology: a Hieroglyphic Transformer for Translating Ancient Egyptian

Mattia De Cao, Nicola De Cao, Angelo Colonna, Alessandro LenciACL 2024

TL;DR

This paper introduces a novel Hieroglyphic Transformer model, adapted from the M2M-100 framework, to translate ancient Egyptian hieroglyphs into German and English. By curating a new dataset from the Thesaurus Linguae Aegyptiae and rigorously cleaning it, the authors demonstrate promising translation accuracy, paving the way for computational approaches to deciphering ancient languages.

What Problem Does This Paper Address?

Ancient Egypt left behind a rich legacy of texts and inscriptions, yet the field of Egyptology has been slow to adopt computational linguistics for translation. Existing efforts primarily focus on optical character recognition (OCR) of hieroglyphs, rather than their translation. This gap highlights a need for deeper integration between Egyptology and Natural Language Processing (NLP). The authors aim to bridge this gap by developing an automatic translation system for ancient Egyptian hieroglyphs, a task that is challenging due to the unique nature of the script and the scarcity of readily usable digital resources.

Methods

The core of this work is the development of a "Hieroglyphic Transformer." This model is built upon the powerful M2M-100 multilingual translation framework, which is pre-trained on 100 modern languages. The authors adapt this existing model for the specific task of translating ancient Egyptian. Key methodological contributions include:

  • Dataset Curation: A new dataset was constructed by extracting and meticulously cleaning data from the Thesaurus Linguae Aegyptiae (TLA) database. This involved significant data filtering, cleaning, and structuring to ensure compatibility with the translation model.
  • Transfer Learning: The researchers leverage transfer learning by fine-tuning the pre-trained M2M-100 model. This allows them to benefit from the model's existing knowledge of language structures and translation patterns, applying it to the low-resource domain of ancient Egyptian.
  • Input Representation: The model can accept hieroglyphs either as Gardiner codes (a standardized alphanumeric system for hieroglyphic signs) or as transliterations (alphabetical representations of hieroglyphic sounds).
  • Evaluation: Both automatic metrics (SacreBLEU and RougeL) and human evaluation were employed to assess the model's performance.

Datasets

The primary dataset used in this research is derived from the Thesaurus Linguae Aegyptiae (TLA) database. The authors created a new, curated dataset from a snapshot of the TLA collected in 2018. This dataset was specifically filtered and cleaned for the purpose of training their Hieroglyphic Transformer. The dataset includes various elements such as Gardiner codes, transliterations, translations (German and English), lemma IDs, and part-of-speech tags. The authors also mention using a validation and test set, each comprising 100 distinct sources.

Key Insights and Ideas

  • The Power of Transfer Learning for Ancient Languages: The success of adapting a model trained on modern languages (M2M-100) to ancient Egyptian demonstrates the potential of transfer learning for low-resource historical languages.
  • Data Quality is Paramount: The paper strongly emphasizes the impact of data cleaning. Their experiments show a dramatic improvement in translation quality (SacreBLEU from 4.0 to 54.4) after implementing their data cleaning procedures, highlighting that even with powerful models, raw data can be a significant bottleneck.
  • Hieroglyphic vs. Transliteration Input: While both input formats were tested, transliteration generally yielded higher translation accuracy, suggesting that the phonetic representation might be more directly usable by the Transformer architecture than the visual hieroglyphic codes.
  • The Value of Diverse Data Pairs: The "ALL" model, which incorporates various combinations of source and target data (including German, English, transliteration, and POS tags), achieved the best or second-best performance across most metrics. This indicates that richer input signals contribute to a more robust translation model.

Key Results

The Hieroglyphic Transformer achieved promising results in translating ancient Egyptian hieroglyphs into German and English.

  • Automatic Evaluation: The "ALL" model (trained with all available data pairs) performed best or second-best across various configurations, achieving SacreBLEU scores up to 86.4 and RougeL scores up to 88.4 in specific configurations.
  • Impact of Data Cleaning: Cleaning the raw data significantly boosted performance, increasing SacreBLEU from 4.0 to 54.4 and RougeL from 18.4 to 62.8 for the German translation task.
  • Human Evaluation: The model performed well on short to medium-length input texts (one to two sentences). It demonstrated effectiveness in generating outputs for transliteration, POS tags, and lemma IDs. However, it showed some inconsistencies with longer texts (exceeding three clauses) and rare terms.

Limitations and Future Work

  • Limited Language Phases: The current model is primarily trained on Ancient and Middle Egyptian. It may encounter challenges with later linguistic phases or grammatical structures.
  • Input Length Sensitivity: The model struggles with very long sentences, particularly when they exceed three clauses.
  • Rare Terms: Performance degrades when encountering rare terms in the input.
  • Future Work: The authors suggest expanding the dataset to include other Egyptian language phases (Late Egyptian, Demotic, Coptic), integrating more modern languages, and conducting more comprehensive experiments. They also propose further research into the potential benefits of using Arabic as an input language due to perceived similarities in sentence construction.

Summary updated

BibTeX
@inproceedings{cao2024deep,
  author = {Mattia De Cao and Nicola De Cao and Angelo Colonna and Alessandro Lenci},
  title = {{Deep Learning Meets Egyptology: a Hieroglyphic Transformer for Translating Ancient Egyptian}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure