TL;DR
This paper investigates how morphological and lexical features impact dependency parsing for Persian, a morphologically rich language. By experimenting with two state-of-the-art parsers (MaltParser and MSTParser) and a new Persian dependency treebank, the authors establish baselines and demonstrate that incorporating these features significantly improves parsing accuracy, especially when dealing with data sparsity.
What Problem Does This Paper Address?
Dependency parsing aims to determine the grammatical relationships between words in a sentence, often represented as a tree structure. While data-driven approaches have shown promise, their performance can vary significantly depending on the language's characteristics. Morphologically rich languages (MRLs), like Persian, present a particular challenge because they encode a lot of grammatical information (like tense, mood, person, number) directly within word forms, leading to a large number of word variations. This richness, combined with relatively free word order, makes it difficult for parsers to accurately identify dependencies. Existing parsers often don't leverage language-specific morphological and lexical information effectively, leading to suboptimal performance. This paper addresses this gap by empirically studying the impact of various such features on Persian dependency parsing, aiming to find optimal configurations and strategies for MRLs.
Methods
The authors employ two popular data-driven dependency parsers: MaltParser (transition-based) and MSTParser (graph-based). They conduct a series of experiments to evaluate the impact of different feature sets and configurations. Key methodological aspects include:
- Baseline Establishment: They first establish baseline performance for both parsers using default settings and then optimize them through parameter tuning and algorithm selection.
- Feature Engineering: They explore a range of features, including:
- Treebank-annotated features: Attachment, Animacy, Number, Person, Comparison, and Tense/Mood/Aspect (TMA).
- Automatic Semantic Features: Word Clusters (WC) derived from Brown clustering and Semantic Verb Clustering (VC).
- Lexical Ontology Features: Synset Identifier (SID) from FarsNet and Semantic File (SF).
- Derived Features: Tense (T) and Mood (M) were derived from TMA values.
- Data Sparsity Reduction: To combat the out-of-vocabulary problem common in MRLs, they investigate three sparsity reduction techniques: replacing word forms with lemmas, number normalization, and using word clusters/semantic files.
- Experimental Setup: They use a standard train/development/test split (80-10-10) and evaluate performance using Unlabeled Attachment Score (UAS), Labeled Attachment Score (LAS), and Label Accuracy (LA). Statistical significance is tested using McNemar's test.
Datasets
The primary dataset used is the Persian dependency treebank version 1.0 (Rasooli et al., 2013), which is a newly released resource containing approximately 30,000 sentences and half a million tokens. This treebank is annotated with syntactic roles and morpho-syntactic features. The authors also mention the UPDT (Uppsala Persian Dependency Treebank) for comparison of statistical properties. For POS tagging, they utilize the Bijankhan corpus and a merged dataset created by the PLP Toolkit (Sarabi et al., 2013).
Key Insights and Ideas
- Morphology Matters: The study strongly confirms that incorporating morphological and lexical features is crucial for achieving high accuracy in Persian dependency parsing.
- Feature Combinations are Key: While individual features show varying degrees of impact, specific combinations of morphological and semantic features yield the best results, often a single morphological feature combined with a single semantic feature.
- Data Sparsity is a Major Hurdle: For MRLs like Persian, data sparsity is a significant challenge. Replacing word forms with lemmas and number normalization are effective strategies to mitigate this.
- Parser Differences: MSTParser (graph-based) generally achieves higher overall accuracy than MaltParser (transition-based), especially in gold settings. However, MaltParser appears more robust to noisy (predicted) information.
- Representation Choice Impacts Performance: The two representations of the Persian treebank (manual vs. automatically converted) show a performance difference, with the manually annotated version (which explicitly marks the accusative case marker) performing better.
Key Results
- The final achieved LAS scores were 87.91% for MaltParser and 88.37% for MSTParser on the test set, representing improvements of 0.23% and 0.39% over their respective baselines.
- The best performing feature combination for MaltParser was {Mood, Semantic File}, while for MSTParser it was {TMA, Word Clusters}.
- Sparsity reduction techniques improved accuracy: replacing word forms by lemma improved MaltParser by 0.38% and MSTParser by 0.06%. Number normalization improved MaltParser by 0.01% and MSTParser by 0.05%.
- Using predicted POS tags and morphological information led to a drop in accuracy (e.g., 0.72% for MaltParser, 1.23% for MSTParser), highlighting the importance of accurate input features.
Limitations and Future Work
- Limitations:
- The study acknowledges that the current TMA values in the treebank are not sufficient to fully derive tense and mood, forcing the use of TMA directly in predicted settings.
- The performance of both parsers degrades on longer sentences due to the limited number of long sentences in the treebank.
- The paper uses existing parsers as black boxes, focusing on input data and feature engineering rather than modifying the parsing algorithms themselves.
- Future Work:
- The authors plan to explore modifications to the underlying parsing algorithms to better leverage morphological information.
- Further investigation into the optimal granularity of semantic features is suggested.
- Developing more robust methods for handling data sparsity and out-of-vocabulary words in MRLs remains an open area.
Summary updated