TL;DR
This paper tackles the challenge of cross-lingual dependency parsing for closely related languages by exploring annotation projection and treebank translation. Their approach surprisingly outperforms even fully supervised models for Slovak, demonstrating the power of transfer learning for low-resource languages.
What Problem Does This Paper Address?
Building robust natural language processing tools for every language is a monumental task, especially for dependency parsing, which requires extensive annotated data. This paper addresses the challenge of creating dependency parsers for languages with limited resources by leveraging existing treebanks from related languages. The core idea is that if languages are linguistically similar, knowledge gained from a well-resourced language can be transferred to a less-resourced one, significantly reducing the need for manual annotation. This is particularly relevant for "closely related languages" where syntactic structures and vocabulary often overlap, making them ideal candidates for such transfer learning.
Methods
The authors employ two primary methods for cross-lingual transfer:
- Annotation Projection: This technique involves aligning parallel sentences (e.g., from movie subtitles) and projecting the dependency parse structure from the source language to the target language. They utilize heuristics to handle non-one-to-one alignments, including the use of "dummy nodes" which are later removed.
- Treebank Translation: This approach involves translating entire source language treebanks into the target language using machine translation (both phrase-based and syntax-based SMT). The dependency annotations are then transferred to the translated sentences.
A key aspect of their work is the evaluation of adding target language POS and morphological tags to the projected data, which can either supplement or overwrite the projected information. They compare these methods against baseline delexicalized models and fully supervised models trained on target language data.
Datasets
The paper uses data from the VarDial 2017 shared task, which includes parallel corpora (specifically movie subtitles) for training and development, and treebanks for several closely related languages:
- Target Languages: Croatian, Slovak, Norwegian, Danish
- Source Languages: Slovenian, Czech, Swedish
The specific sizes of projected data used are mentioned as 100,000, 500,000, and 1,000,000 sentence pairs.
Key Insights and Ideas
- Slovak as a Surprise Success: The most striking finding is that their cross-lingual models for Slovak outperform the fully supervised models trained on Slovak data. This is attributed to the large size of the Czech treebank and the close linguistic relationship between Czech and Slovak.
- Danish vs. Swedish for Norwegian: Surprisingly, Swedish proved to be a better source language for Norwegian than Danish, despite Danish being linguistically closer. This suggests that subtle annotation differences or domain mismatches can have a significant impact.
- Morphological Tagging is Crucial: Adding morphological information to projected data consistently improves performance, especially for highly inflecting languages like Slovak and Croatian.
- LAS vs. UAS Gap: A notable difference between labeled attachment scores (LAS) and unlabeled attachment scores (UAS) suggests that while the models can correctly identify dependency relations, accurately labeling them is a challenge. This gap is larger than in fully supervised models.
Key Results
- Outperforming Supervised Models: For Slovak, the best cross-lingual model achieved a higher Labeled Attachment Score (LAS) than the fully supervised baseline.
- General Improvement: All cross-lingual models significantly outperformed the delexicalized baselines across all language pairs.
- Treebank Translation Advantage: For most language pairs, treebank translation methods (PBSMT and SyntaxSMT) generally performed better than annotation projection, though the differences were often small.
- Data Size Matters: Contrary to previous findings, the authors observed increasing performance with larger amounts of projected data (up to 1 million sentence pairs), likely due to the use of movie subtitles which are generally shorter and potentially more domain-mismatched than parliamentary proceedings.
Limitations and Future Work
- LAS vs. UAS Discrepancy: The significant gap between LAS and UAS is acknowledged as a limitation, suggesting that language-specific dependency labels, which are not directly projected, might be a contributing factor. Further investigation into this is needed.
- Croatian's Difficulty: Croatian remained a challenging target language, with improvements over the baseline being modest.
- Danish/Swedish Anomaly: The unexpected performance of Danish as a source for Norwegian warrants further investigation into potential annotation differences or domain mismatches.
- Future Work: The authors suggest deeper investigation into the LAS/UAS gap and exploring how to better handle language-specific dependency labels. They also highlight the potential for combining data from multiple source languages.
Summary updated