Development of a Persian Syntactic Dependency Treebank

Mohammad Sadegh Rasooli, Manouchehr Kouhestani, Amirsaeid MoloodiNAACL 2013

TL;DR

This paper introduces the first comprehensive Persian syntactic dependency treebank, a crucial resource for advancing Persian Natural Language Processing (NLP). The treebank, containing approximately 30,000 annotated sentences, addresses the scarcity of high-quality linguistic data for Persian and offers a valuable tool for research and education, particularly due to its rich verb lemma coverage.

What Problem Does This Paper Address?

Developing robust NLP systems for any language requires large, well-annotated datasets. Persian, spoken by over 110 million people, is considered a "low-resourced" language in terms of available annotated linguistic data. This scarcity hinders progress in areas like machine translation, parsing, and language learning tools. Specifically, the paper tackles the challenge of creating a syntactic dependency treebank for Persian, which involves complex annotation processes and the need to represent the language's unique linguistic features, such as its relatively free word order and complex verb structures.

Methods

The authors employed a bootstrapping approach to construct the treebank. This involved:

  • Initial Data Collection: Gathering a large corpus of independent Persian sentences from various sources, including news texts.
  • Automated Processing: Utilizing existing Persian NLP tools for tasks like spell checking, tokenization, and Part-of-Speech (POS) tagging. A dependency parser (MST parser) was then used to generate initial dependency trees.
  • Manual Correction: A team of native Persian speakers, including linguists and graduate students, meticulously corrected the automatically generated annotations. This manual review was crucial for ensuring accuracy and refining linguistic conventions.
  • Iterative Improvement: The parser model was periodically retrained on the newly corrected data, allowing for continuous improvement of the automated parsing step.
  • Dual Representation: For object-verb relations, two representations were created: one where the accusative case marker is the head, and another where the direct object is the head. This was done to explore trade-offs between ease of parsing and the number of non-projective arcs.

Datasets

  • Newly Introduced Dataset: The primary contribution is the Persian Syntactic Dependency Treebank, containing approximately 30,000 manually annotated sentences.
  • Existing Resources: The project leveraged the valency lexicon of Persian verbs (Rasooli et al., 2011c) and the Bijankhan corpus (Bijankhan, 2004) for inspiration and comparison.
  • Tools Used: The Virastyar library (Kashefi et al., 2010) for spell checking and tokenization, and the MST parser (McDonald et al., 2005) for dependency parsing were utilized.

Key Insights and Ideas

  • Dependency Grammar Suitability: The paper argues for dependency grammar's effectiveness in representing Persian's free word order, a characteristic shared with languages like Turkish and Czech.
  • Verb Richness: The treebank's inclusion of nearly 5000 distinct verb lemmas is highlighted as a significant asset for educational purposes and linguistic research.
  • Bootstrapping Efficiency: The bootstrapping approach, combining automated tools with manual correction, proved to be an effective strategy for building a large-scale treebank within a reasonable timeframe.
  • Annotation Challenges: The process revealed specific challenges in Persian NLP, such as handling compound verbs, separable verbs, and the nuances of verb inflections, which required careful annotation guidelines.
  • Dual Representation Trade-offs: Creating two representations for object-verb relations allowed for an empirical comparison of their impact on parsing and tree structure.

Key Results

  • Treebank Size: The final treebank comprises 29,982 manually annotated sentences.
  • Linguistic Coverage: The treebank includes detailed annotations for syntactic roles and morphosyntactic features, with a notable emphasis on verb lemmas (4,782 distinct verb lemmas).
  • Annotation Quality: The annotator agreement statistics (reported in Table 7) indicate a high level of agreement, with Kappa values considered "almost perfect" according to established metrics.
  • Non-Projectivity: Statistics on non-projective arcs (Table 5) show a reduction in the second representation, suggesting potential benefits for parsing.
  • Error Correction: The process of manual correction led to a small but significant percentage of changes in labeled relations (6.29%) and POS tags (4.23%), underscoring the value of the double-checking phase.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while doubly annotated corpora are more reliable, annotating each sentence only once (with an additional checking phase) was found to be more cost-effective. The quality of the annotation was estimated using a 5% sample.
  • Colloquial words were removed from the dataset, which might limit its applicability for certain types of NLP tasks.
  • The paper mentions that the POS tagset used is not identical to that of the Bijankhan corpus, requiring conversion for joint use.
  • Future Work:
    • The authors express a desire to add a semantic level of annotation to the treebank in the future.
  • They also plan to publish more details on the linguistic aspects and findings from parsing experiments.
  • The development of desktop and web-based tools for searching and visualizing the treebank is also mentioned as a completed effort.

Summary updated

BibTeX
@inproceedings{rasooli2013development,
  author = {Mohammad Sadegh Rasooli and Manouchehr Kouhestani and Amirsaeid Moloodi},
  title = {{Development of a Persian Syntactic Dependency Treebank}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2013},
}
Figure