Annotating and Disambiguating the Discourse Usage of the Enclitic dA in Turkish

Ebru Ersöyleyen, Deniz Zeyrek, Fırat ÖterACL 2023

TL;DR

This paper tackles the challenge of disambiguating the discourse connective usage of the Turkish enclitic "dA". By creating a novel annotated dataset and employing machine learning with simple linguistic features, they achieve promising accuracy in distinguishing "dA" as a discourse connective from its other grammatical roles. This work contributes to a deeper understanding of discourse connectives, particularly in morphologically rich languages like Turkish.

What Problem Does This Paper Address?

Discourse connectives are crucial for understanding the structure and flow of text, signaling relationships like causality, expansion, or contrast between different parts of a discourse. While many connectives are readily identifiable (e.g., "and," "but," "because"), some linguistic elements, like clitics, can also function as discourse connectives, leading to ambiguity. The Turkish enclitic "dA" is one such element. It's known to be focus-associated and can convey various senses, including additive and contrastive ones. However, its discourse connective usage has not been systematically annotated in existing Turkish corpora, and its disambiguation has been a challenge for natural language processing (NLP) tasks. This paper aims to fill this gap by providing a method to reliably annotate and disambiguate the discourse connective role of "dA" from its other grammatical functions.

Methods

The authors employed a two-pronged approach: annotation and machine learning.

  1. Annotation: They created a new dataset by combining samples from existing Turkish corpora (TDB 1.1 and TS Corpus v2). The annotation followed the principles of the Penn Discourse Treebank (PDTB), focusing on identifying instances of "dA" as a discourse connective (DC) versus non-discourse connective (NDC). Two independent native speakers annotated the data, and inter-annotator agreement was measured using standard metrics (Precision, Recall, F1) and Cohen's kappa, indicating substantial agreement. Disagreed cases were resolved through discussion to create a gold-standard dataset.
  2. Machine Learning: They developed binary classifiers to distinguish between DC and NDC usages of "dA". The approach utilized a canonical set of simple linguistic features, including:
  3. Part-of-Speech (POS) tags: The POS tags of words within a [-3, +3] window around "dA".
  4. Lemmas: The lemmas of words within the same [-3, +3] window.
  5. Proper Noun Indicator: A binary feature indicating whether a word in the preceding window is a proper noun.

    These features were chosen for their ability to capture local cues. To handle Turkish's rich morphology, lemmatization was applied to reduce noise. Several supervised binary classifier models were trained and compared: Logistic Regression, Support Vector Machine (SVM), and Random Forest.

Datasets

  • TDB 1.1 (Turkish Discourse Bank): An existing annotated corpus of explicit and non-explicit discourse connectives.
  • TS Corpus v2: Another Turkish corpus used for extracting "dA" samples.

The authors created a new dataset, referred to as the dA Corpus, by combining samples from TDB 1.1 and TS Corpus v2. This new corpus is specifically designed for annotating and disambiguating the discourse and non-discourse connective usages of "dA". The paper notes that the dA Corpus is available at https://github.com/TurkishdA/dA-Corpus.

Key Insights and Ideas

  • "dA" as a Discourse Connective is Annotatable: Despite "dA"'s ability to cliticize to various syntactic types and its variable position, the authors demonstrate that its discourse connective usage can be reliably annotated. This is a significant step towards computational analysis of such elements.
  • Local Cues are Sufficient for Disambiguation: The study shows that even a minimal set of simple linguistic features (POS tags, lemmas) can effectively disambiguate the discourse connective usage of "dA". This suggests that local context plays a strong role.
  • POS Tags are Highly Predictive: The analysis revealed that POS tags are the most predictive features for disambiguating "dA"'s usage, highlighting the importance of syntactic context.
  • Morphological Richness is a Challenge: Turkish's agglutinative nature, while providing rich semantic information, can also introduce noise for NLP models due to extensive morphological variation. Lemmatization proved to be a useful technique to mitigate this.
  • Error Analysis Points to POS Tagging Issues: A significant source of classification errors was attributed to incorrect POS tag assignments, particularly for nominal/adjectival predicates, indicating areas for improvement in upstream NLP tools for Turkish.

Key Results

The machine learning models achieved an average accuracy of 0.77 in disambiguating the discourse connective usage of "dA". The best performing model, a combination of POS tags, lemmas, and proper nouns, reached an accuracy of 0.82. The study also found that POS tags alone achieved a minimum accuracy of 0.76, underscoring their predictive power.

Limitations and Future Work

  • Corpus Size: The authors acknowledge that the size of the created corpus is a limitation. They suggest that a larger dataset could potentially reveal more linguistic features of discourse connectives and improve performance.
  • Novelty of Features: While the linguistic features used are not entirely novel, the paper emphasizes that achieving promising results with a minimal set is a key contribution.
  • Discourse Senses: The annotation of specific discourse senses of "dA" (beyond just DC/NDC) was outside the scope of this work but is suggested as a direction for future research.
  • Generalizability: The authors hope their work will serve as a benchmark for Turkish clitics and a reference point for similar studies in other languages that feature clitics with discourse functions. Further work could explore more complex feature sets and investigate the role of "dA" in cross-lingual discourse analysis.

Summary updated

BibTeX
@inproceedings{ersoyleyen2023annotating,
  author = {Ebru Ersöyleyen and Deniz Zeyrek and Fırat Öter},
  title = {{Annotating and Disambiguating the Discourse Usage of the Enclitic dA in Turkish}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}

Loading related content...

Figure