Revisiting and Amending Central Kurdish Data on UniMorph 4.0

Sina Ahmadi, Aso MahmudiACL 2023

TL;DR

This paper identifies significant inaccuracies and limitations in the existing UniMorph 4.0 dataset for Central Kurdish, proposing a new, more comprehensive dataset and a methodology for its creation. The work highlights the critical need for high-quality linguistic data to improve computational morphology tools, especially for under-resourced languages.

What Problem Does This Paper Address?

Computational morphology, the study of word formation, is crucial for natural language processing (NLP) tasks like parsing and machine translation. The UniMorph project aims to provide standardized morphological data for many languages. However, this paper reveals that the Central Kurdish (Sorani) data in UniMorph 4.0 suffers from several issues: it's incomplete, contains systematic errors in morphemes and morphotactics, and uses an unconventional writing system. These problems hinder the development of accurate NLP tools for Central Kurdish. The authors aim to address this by analyzing the existing data's flaws and proposing a more reliable and comprehensive dataset.

Methods

The authors employ a multi-pronged approach:

  • Data Analysis: They meticulously analyze the existing UniMorph 4.0 Central Kurdish data, identifying specific errors in morphemes, morphotactics, and writing conventions.
  • Linguistic Feature Identification: They pinpoint specific linguistic phenomena in Central Kurdish, such as the Izafe construction and various clitics, that were inadequately represented or misrepresented in the UniMorph data.
  • Finite-State Transducers (FSTs): They leverage existing FSTs for Central Kurdish (Naserzade et al., 2023) to develop a morphological analyzer and generator. This allows them to systematically create a wide range of inflected forms.
  • Dataset Creation: They generate two new datasets:
  • Gold-standard: 1,000 words extracted from a corpus and manually verified for accuracy and context.
  • Silver-standard: Full paradigms for a selection of nouns, adjectives, and verbs, generated without context but aiming for comprehensive coverage.
  • Orthographic Handling: Their new dataset is provided in both the conventional Perso-Arabic script and a Latin-based script, facilitating comparisons with other Kurdish dialects.

Datasets

  • UniMorph 4.0 Central Kurdish Dataset: Used as the baseline and for identifying issues. Contains 24,316 word-forms.
  • New Gold-standard Dataset: 1,000 word-forms extracted from a corpus and manually verified.
  • New Silver-standard Dataset: Contains full paradigms for 10 nouns, 5 adjectives, 17 intransitive verbs, and 12 transitive verbs.
  • Table A.1: Lists selected words for which full paradigms were generated.

Key Insights and Ideas

  • Data Quality is Paramount: Even with advanced NLP approaches, the performance of morphological analysis and generation systems is fundamentally limited by the quality and coverage of the underlying linguistic data.
  • UniMorph's Limitations for Central Kurdish: The UniMorph schema and its application to Central Kurdish have significant shortcomings, including an unconventional writing system and a lack of representation for crucial linguistic features like Izafe.
  • The Importance of Orthography: The use of an unconventional Latin-based script for Central Kurdish in UniMorph is a major impediment, as it doesn't reflect how speakers actually write and omits important phonemes.
  • Morphotactic Complexity: Central Kurdish exhibits complex morphotactic rules, particularly in verb conjugation, which were not consistently captured in the UniMorph data, likely due to influence from related languages.
  • Synthesis Degree: Central Kurdish demonstrates a relatively high degree of synthesis, especially in adjectives and verbs, indicating a rich inflectional system that requires careful modeling.

Key Results

  • Error Estimation: The authors estimate that approximately 25% of the forms in the current UniMorph 4.0 Central Kurdish dataset are incorrect.
  • Morphological Reinflection Accuracy: A non-neural baseline model achieved an accuracy of 48.7% on the UniMorph 4.0 data. Their newly created datasets showed improved accuracy (63.5% to 67.5% for Gold-standard, 61.2% to 65.0% for Silver-standard) and better average edit distance.
  • Dataset Size: The UniMorph 4.0 dataset contains 24,020 total forms. Their proposed datasets are significantly larger, with the Silver-standard dataset alone containing over 110,000 forms.
  • Synthesis Degree: Central Kurdish exhibits a synthesis degree of 2.22, comparable to languages like Old English and Swahili.

Limitations and Future Work

  • Dataset Scope: The new datasets primarily focus on inflectional morphology and cover a limited range of close-class parts of speech (nouns, verbs, adjectives). Derivational morphology and word formation are not extensively covered.
  • Dialectal Variation: The current work focuses on the standard variety of Central Kurdish (Sulaymaniyah). Future work should include other dialects.
  • Lack of Treebank: The absence of a Central Kurdish treebank makes it challenging to bridge morphology and syntax.
  • UniMorph Schema: The authors suggest that the UniMorph schema's use of generic LGSPEC tags for language-specific features is not ideal and advocate for more specific tags or a more advanced schema for languages with rich morphology.
  • Future Work: The authors plan to extend their work to include other varieties of Central Kurdish, derivational morphology, and to bridge morphology and syntax. They also propose that their new dataset be incorporated into future versions of UniMorph.

Summary updated

BibTeX
@inproceedings{ahmadi2023revisiting,
  author = {Sina Ahmadi and Aso Mahmudi},
  title = {{Revisiting and Amending Central Kurdish Data on UniMorph 4.0}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure