UD-ETCSUX: Toward a Better Understanding of Sumerian Syntax

Kenan Jiang, Adam AndersonACL 2024

TL;DR

This paper introduces UD-ETCSUX, the first dependency dataset for Sumerian, a historically challenging language. They developed a dependency parser for Sumerian transliterations, demonstrating the potential of Universal Dependencies for ancient languages and highlighting key challenges in Sumerian syntax.

What Problem Does This Paper Address?

Sumerian, one of the earliest known written languages, presents a significant challenge for linguistic analysis due to its complex morphology and agglutinative nature. While numerous grammars exist, they largely omit detailed rules for dependency grammar, which describes the grammatical relationships between words in a sentence. This gap makes it difficult to apply modern computational linguistics techniques to understand Sumerian syntax. Existing computational tools primarily focus on lemmatization and part-of-speech tagging, leaving dependency parsing largely unexplored. This paper aims to bridge this gap by developing a dependency parser for Sumerian transliterations within the Universal Dependencies (UD) framework.

Methods

The authors developed a dependency parser for Sumerian transliterations by leveraging the Universal Dependencies (UD) framework. Their methodology involved:

  • Dataset Creation (UD-ETCSUX): They extracted Sumerian transliterations and their English translations from existing corpora like ETCSRI and ETCSL. They then used spaCy to perform dependency tagging on the English translations and manually transferred these UD labels to the corresponding Sumerian transliterations.
  • Compound Verb Detection: To address the complexity of Sumerian compound verbs, they developed a detector that identifies and categorizes these verbs, aiding in more accurate parsing.
  • Word Embeddings: They trained two types of word embedding models (FastText and Pointwise Mutual Information - PMI) on both lemmas and transliterations from large Sumerian corpora to capture semantic relationships.
  • Dependency Parsing Implementation: They built their dependency parser using spaCy's training framework, tokenizing Sumerian by spaces. They employed a 10-fold cross-validation for evaluation and used techniques like dropout to prevent overfitting.
  • Data Augmentation: To mitigate the challenges of a small dataset, they used data augmentation by replacing words with semantically similar ones, generating new training examples.

Datasets

  • ETCSRI (Sumerian Royal Inscriptions): Used for extracting Sumerian transliterations and English translations.
  • ETCSL (Electronic Text Corpus of Sumerian Literature): Used for extracting Sumerian transliterations and English translations.
  • UD-ETCSUX: A newly introduced dataset containing dependency trees for 133 Sumerian transliterations, with a total of 573 labeled data points. This dataset is intended to be compatible with the Universal Dependencies framework.

Key Insights and Ideas

  • UD for Ancient Languages: The paper demonstrates that the Universal Dependencies framework, widely used for modern languages, can be effectively adapted for ancient languages like Sumerian, opening doors for cross-linguistic NLP research.
  • Compound Verbs as a Major Hurdle: Sumerian's complex and varied compound verbs pose a significant challenge for accurate dependency parsing, necessitating specialized tools like the one developed in this paper.
  • Data Augmentation's Promise: Even with a limited dataset, data augmentation techniques show promise in improving parsing performance for low-resource ancient languages.
  • The Need for Collaboration: The authors emphasize the critical need for collaboration between computational linguists and Sumerologists to advance the development of NLP tools for Sumerian.

Key Results

  • The UD-ETCSUX dataset was created, containing 133 Sumerian transliterations with dependency annotations.
  • A dependency parser was developed, achieving a highest Unlabeled Attachment Score (UAS) of 51.54 and a highest Labeled Attachment Score (LAS) of 13.75 (after data augmentation).
  • FastText embeddings trained on lemmas yielded the best performance in terms of UAS.
  • Data augmentation with 60 additional examples led to consistent improvements in both UAS and LAS scores.
  • Qualitative evaluation showed the parser's ability to correctly identify nominal subjects, objects, and compound particles in straightforward sentences. However, it also highlighted instances of misclassification, particularly with adjectival modifiers.

Limitations and Future Work

Limitations:

  • Limited Dataset Size: The current dataset of 133 transliterations is small, which limits the parser's ability to explore diverse linguistic scenarios and achieve higher accuracy.
  • Expert Validation: More extensive expert validation is required to enhance the dataset's robustness.
  • Misclassification of Adjectival Modifiers: The parser struggled with correctly differentiating adjectival modifiers from nominal subjects in certain cases, indicating a need for more complex sentence structures in the training data.

Future Work:

  • Expand UD-ETCSUX: Incorporate feedback from language experts and continuously expand the dataset to improve its quality and coverage.
  • Inter-annotator Agreement Studies: Conduct studies to ensure the reliability and consistency of annotations.
  • Address Specific Issues: Target solutions for identified issues, such as the accurate handling of adjectival modifiers and the inclusion of multiple dependency parsings for polysemous sentences.
  • Publish in UD Treebank: Aim to publish the UD-ETCSUX dataset in the Universal Dependency Treebank.
  • Annotate Sub-word Particles: Provide full annotations for sub-word particles in Sumerian texts to enable automated dependency parsing.

Summary updated

BibTeX
@inproceedings{jiang2024ud,
  author = {Kenan Jiang and Adam Anderson},
  title = {{UD-ETCSUX: Toward a Better Understanding of Sumerian Syntax}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure