TL;DR
This paper introduces the first automatic language and dialect identification system for cuneiform texts, a historically significant but computationally challenging domain. By creating a new dataset and adapting existing machine learning techniques, the authors demonstrate the feasibility of this task, paving the way for deeper computational analysis of ancient languages.
What Problem Does This Paper Address?
Cuneiform, one of the earliest writing systems, presents a unique challenge for computational linguistics. While vast amounts of cuneiform texts exist, they are often in transliterated form, and the languages and dialects within them are complex and closely related. Traditional methods for language identification, which rely on large, well-resourced corpora and distinct linguistic features, struggle with the limited data, script variations, and close relationships between Sumerian and various Akkadian dialects. This paper tackles the problem of automatically identifying the language and dialect of cuneiform texts, a crucial step for unlocking their potential for large-scale computational analysis, historical research, and digital humanities applications.
Methods
The authors leverage the HeLI (Heuristic Language Identification) method, a supervised learning approach that uses sign n-grams (sequences of characters) to build language models. For cuneiform, "signs" are treated as characters, and the system analyzes sequences of these signs within transliterated text. They also implemented and evaluated four baseline language identifiers:
- Simple Scoring: Assigns a score of 1 for each n-gram found in a language model.
- Sum of Relative Frequencies: Adds the relative frequencies of n-grams to a language's score.
- Product of Relative Frequencies: Multiplies the relative frequencies of n-grams.
- Majority Voting Ensemble: Combines the predictions of the three previous methods.
A key methodological innovation is the development of Nuolenna, a tool to transform transliterated cuneiform text back into its original Unicode cuneiform script. This allows for experiments that more closely mimic the original written form.
Datasets
The primary dataset used in this research is derived from the Open Richly Annotated Cuneiform Corpus (Oracc). This corpus contains transliterated cuneiform texts.
- Oracc Corpus: The source material, comprising over 13,000 texts.
- CLI Dataset: A subset extracted from Oracc for the Cuneiform Language Identification (CLI) 2019 shared task. This dataset was divided into training, development, and test portions.
- Preliminary Experiments Dataset: Derived from Oracc, this dataset was used for initial experiments and included both in-domain and out-of-domain settings, applied to individual lines and entire texts.
- CLI Shared Task Dataset: A specially tailored dataset for the shared task, with specific sizes for development (668 lines per language/dialect) and test sets (985 lines per language/dialect) after filtering out very short lines.
Key Insights and Ideas
- Feasibility of Automatic Identification: The research demonstrates that automatic language and dialect identification is indeed possible for cuneiform texts, even with their inherent complexities.
- N-grams as Effective Features: Sign n-grams, when applied to transliterated cuneiform, prove to be effective features for distinguishing between languages and dialects.
- In-domain vs. Out-of-domain Performance: As expected, language identification performs significantly better in in-domain settings (where training and test data are similar) compared to out-of-domain settings.
- Text Length Matters: Identifying the language of complete texts is considerably easier than identifying it from individual lines, especially short ones.
- Nuolenna's Utility: The development of the Nuolenna tool is crucial for enabling experiments that work with the actual cuneiform script, bridging the gap between transliteration and the original writing system.
Key Results
- The Product of Relative Frequencies method achieved the highest macro F1-score of 0.7206 on the CLI dataset using sign n-grams from 1 to 4.
- The HeLI method also performed well, achieving an F1-score of 0.7171 for lines and 0.7061 for texts in the preliminary experiments.
- In the preliminary experiments, the best F1-score for texts in-domain was 93%, while for lines out-of-domain, it was 60%.
- The confusion matrix revealed that Standard Babylonian and Neo-Babylonian were the most difficult Akkadian varieties to distinguish, often being misclassified as each other. Late Babylonian was the easiest to identify, with a recall of over 96%.
Limitations and Future Work
- Short and Ambiguous Lines: The performance of the identification methods was significantly impacted by very short lines, which often consisted of a single sign and were ambiguous across dialects and languages.
- Data Availability: Some Oracc projects, particularly those with unannotated dialects or limited Sumerian texts, were excluded from the corpus.
- Future Work: The authors suggest several avenues for future research, including:
- Language set identification for multilingual texts.
- Unsupervised clustering of cuneiform data without predefined languages.
- Improving dialect identification to provide more detailed annotations for Oracc.
Summary updated