TL;DR
This paper presents the BASQUE_TEAM system for dependency parsing, which leverages morphological features to improve parsing accuracy, especially for morphologically rich languages. By systematically analyzing the impact of individual morphological features, they identify the most beneficial ones and combine them with state-of-the-art parsers (MaltParser and Mate) to achieve improved performance.
What Problem Does This Paper Address?
Dependency parsing aims to determine the grammatical relationships between words in a sentence. While state-of-the-art parsers perform well on languages with simpler morphology (like English), they struggle with morphologically rich languages (MRLs) such as Basque, Turkish, and Greek. These languages have complex word structures with many inflections and affixes that carry significant grammatical information. Previous approaches often included all available morphological features without understanding their individual impact, sometimes leading to suboptimal results or even negative effects. This paper addresses the challenge of effectively utilizing morphological information in dependency parsing for MRLs by systematically evaluating the contribution of each feature.
Methods
The BASQUE_TEAM system employs a multi-stage approach:
- Feature Analysis: The core innovation is a systematic analysis of each morphological feature within the
FEATScolumn of the Universal Dependencies (UD) format. They use the Mate parser to evaluate the impact of each feature individually, starting with an emptyFEATScolumn and then adding one feature at a time. This allows them to quantify the contribution of each feature to parsing accuracy. - Feature Selection and Manipulation: Based on the analysis, they identify the most significant features. They then create several parser variants:
- Using only the top 3 most relevant features.
- Moving the most relevant feature to the Part-of-Speech (POS) tag column.
- Moving the most relevant feature to the Coarse-Part-of-Speech (CPOS) tag column.
- They also explore right-to-left versions of these variants.
- Parser Combination: Finally, they combine the outputs of these individual parsers (including baselines) using a voting approach with the MaltBlender tool to create an ensemble system.
Datasets
The experiments were conducted on data from the SPMRL'2013 Shared Task. The authors selected five languages for their experiments due to time constraints and the relevance of their morphological information:
- Basque
- French
- German
- Hungarian
- Swedish
Crucially, the experiments were performed on automatically predicted POS and morphosyntactic tags, reflecting real-world scenarios where gold-standard tags are not always available.
Key Insights and Ideas
- Morphological features are not equally important: The study reveals that while some features (like case in Basque) significantly boost parsing accuracy, others have minimal or even detrimental effects when used in isolation. This challenges the "more is better" approach to feature inclusion.
- Language-specific feature importance: The impact of morphological features varies greatly across languages. For instance, case is highly influential in Basque but less so in Swedish.
- Feature manipulation can be effective: Moving a highly informative feature (like case) to a more prominent column (POS or CPOS) can sometimes yield better results than simply including it in the
FEATScolumn, especially for MaltParser. - Ensemble methods amplify gains: While individual parser variants might show modest improvements, combining them through voting (MaltBlender) can lead to significant gains, particularly for languages where morphological features are highly informative.
Key Results
- Basque: The most sensitive language, showing a 5.7% LAS increase by using "case" alone. Including all features yielded a 6.5% improvement over the baseline.
- French & German: Morphological features had a minimal impact, with small increases (e.g., 0.1% for gender in French).
- Hungarian: Demonstrated significant variation in LAS depending on the feature used, with "case" providing a 5.6% increase. The overall improvement with all features was 7.5%.
- Swedish: Case was not beneficial; "verbform" was the most significant feature.
- Parser Combinations: The combined system achieved notable improvements, with Basque seeing a 3.2% increase over the Mate baseline and Hungarian a 1.3% increase. The best overall LAS for Basque was achieved by a Mate variant using the top 3 features (83.4%).
Limitations and Future Work
- Limited Parser Set: Due to time constraints, only MaltParser and Mate were used. The authors suggest exploring other parsers like MST or EasyFirst.
- Language Scope: The study focused on five languages. Future work could extend this to other MRLs in the SPMRL 2013 task (Korean, Hebrew, Arabic, Polish) and languages like Turkish and Czech, which share characteristics with Basque and Hungarian.
- Feature Engineering: While they explored moving features to POS/CPOS, further experimentation with feature engineering and new parsing algorithms could yield even greater improvements.
- Ensemble Diversity: The authors note that while some variants offer slight improvements, their combination leads to significant gains, suggesting that exploring a wider spectrum of parsers and algorithms for ensemble building could be beneficial.
Summary updated