TL;DR
This paper tackles the challenge of automatically identifying "classifiers" (determinatives) in Ancient Egyptian hieroglyphs, a crucial but under-resourced task. They frame it as a sequence-labeling problem and demonstrate that even with limited data, neural models, particularly a Transformer-based approach (ByT5), can achieve promising results, outperforming simpler baselines and even RNN-based models.
What Problem Does This Paper Address?
Ancient Egyptian writing uses graphemic classifiers, silent hieroglyphic signs that clarify the meaning or pronunciation of a word. Identifying these classifiers is vital for understanding the language and its cultural heritage. However, computational research on Ancient Egyptian is nascent, and there's a lack of tools for automatically identifying these classifiers. This paper aims to bridge this gap by formulating classifier identification as a sequence-labeling task and developing neural models to solve it, treating Ancient Egyptian as a "low-resource" language due to the limited availability of annotated data.
Methods
The authors approach classifier identification as a sequence-labeling task. They preprocess the hieroglyphic transcriptions (using the Manuel de Codage system) by simplifying them to focus on signs and the special tilde markers used for classifiers. They explore two tokenization strategies: character-based (small vocabulary) and sign-based (large vocabulary, one token per unique sign). For output formatting, they opt for a binary labeling scheme (0 for non-classifier, 1 for classifier) which simplifies the task for the models.
They compare three main neural models:
- Character-based 3-layer encoder-decoder LSTM: Processes input as sequences of characters.
- Sign-based 3-layer encoder-decoder LSTM: Processes input as sequences of unique hieroglyphic signs.
- ByT5-small: A Transformer-based model pre-trained on a large corpus of text from various languages.
They also establish several frequency-based baselines (Top-N, CLF-only, CLF-majority) to benchmark their neural models against simpler statistical approaches.
Datasets
The primary dataset used is a subset of the Coffin Texts corpus, which is part of the iClassifier project. This corpus is word-based and includes annotations for both classified and unclassified wordforms, reflecting the linguistic usage of the time. The fully annotated subset contains 74,106 data points, which are then reduced to 8,423 unique wordform types. This dataset is randomly split into 6,739 for training, 842 for development, and 842 for testing.
Additionally, a smaller corpus of Late Egyptian narratives (404 data points) is used as an out-of-domain (OOD) test set to evaluate the models' generalization capabilities. This corpus represents a different textual genre and is from a later period (13th century BCE) compared to the Coffin Texts (22nd-17th c. BCE).
Key Insights and Ideas
- Ancient Egyptian as a Low-Resource NLP Task: The paper successfully frames a linguistic task in an ancient language as a modern NLP problem, demonstrating the applicability of sequence labeling to historical texts.
- Transformer Power in Low-Resource Settings: The ByT5 model, despite being character-based and not specifically trained on Ancient Egyptian, significantly outperforms both RNN-based models and frequency baselines, suggesting that pre-training on diverse data can facilitate knowledge transfer to related, albeit low-resource, domains.
- Importance of Sign-Level Representation: The sign-based LSTM model performs much better than the character-based LSTM, indicating that treating individual hieroglyphic signs as distinct tokens is more effective than treating them as sequences of characters.
- Data Homogeneity: The Coffin Texts corpus appears to be linguistically homogeneous, as evidenced by the minimal performance drop between the development and test sets.
- Challenges of Out-of-Domain Generalization: Performance drops significantly on the Late Egyptian OOD dataset, highlighting the challenges of generalizing to different genres and time periods, even within the same language.
Key Results
The paper reports the average number of misclassified signs per data point.
- In-domain (Coffin Texts):
- ByT5-small: Achieved the lowest error rate at 0.08 on the test set.
- Sign-based LSTM: Performed very competitively with 0.11 errors.
- CLF-majority baseline: Achieved 0.28 errors, showing that even simple methods can be somewhat effective.
- Character-based LSTM: Performed poorly with 0.21 errors, barely beating the CLF-majority baseline.
- Out-of-domain (Late Egyptian Narratives):
- ByT5-small: Achieved 0.35 errors.
- Sign-based LSTM: Achieved 0.38 errors.
- CLF-majority baseline: Achieved 0.49 errors.
- Character-based LSTM: Suffered a performance collapse with 3.07 errors.
Limitations and Future Work
- Limited Data: The primary limitation is the small size of the annotated dataset, which makes training complex models challenging and impacts out-of-domain performance. The authors note that the OOD test set contains new CLFs not seen in training, raising demands on training set size.
- Out-of-Domain Performance: The significant drop in performance on the Late Egyptian corpus indicates that current models struggle to generalize across different genres and time periods.
- Future Work: The authors suggest improving model accuracy and exploring a more fine-grained classification of sign functions, distinguishing between semantic, grammatical classifiers, and phono-repeaters. They also propose investigating knowledge transfer between classifier languages.
Summary updated