Abstract
This paper explores the task of leveraging typology in the context of cross-lingual dependency parsing. While this linguistic information has shown great promise in pre-neural parsing, results for neural architectures have been mixed. Our investigation finds that: 1) the benefit of typological information is derived from coarsely grouping languages into syntactically-homogeneous clusters; 2) typology consistent with actual corpus statistics yields better transfer performance; and 3) typological similarity is only a rough proxy of cross-lingual transferability.
Introduction
Dependency parsing is a critical component of many NLP pipelines, yet it remains a challenge for the majority of the world's languages due to a lack of annotated treebanks. Cross-lingual transfer learning aims to solve this by training models on resource-rich languages and applying them to resource-poor ones. Linguistic typology—which categorizes languages based on structural features like word order—has traditionally been used to bridge the gap between source and target languages. However, in the era of neural parsers, the effectiveness of these features is less clear.
Methodology
We employ a state-of-the-art Biaffine Parser and investigate two primary methods for integrating typological information: Selective Sharing, where typology acts as a bias term, and Input Features, where typological vectors are concatenated to the encoder's input.
We evaluate three distinct typological representations:
- $T_L$ (Linguistic Typology): Features derived from the World Atlas of Language Structures (WALS).
- $T_D$ (Liu Directionalities): Vectors representing the average direction of dependency links within a corpus.
- $T_S$ (Surface Statistics): Proxies for typology based on part-of-speech (POS) tag sequence frequencies.
Results and Analysis
The Clustering Effect
A key finding is that neural parsers utilize typological data primarily for coarse clustering rather than learning fine-grained compositional rules. As shown in Figure 1, languages naturally form clusters based on these features. Experiments replacing high-dimensional vectors with simple cluster IDs (K-Means) showed negligible performance loss, suggesting the model uses typology to identify "neighborhoods" of similar languages.
Performance Comparison
The results across 15 test languages indicate that $T_D$ (Liu Directionalities) provides the most consistent improvements. This is likely because $T_D$ reflects the actual statistical properties of the training corpora, whereas $T_L$ (WALS) represents idealized linguistic labels that may conflict with specific treebank distributions (e.g., the "Arabic Problem" where WALS labels Arabic as SV, but corpora are predominantly VS).
As seen in Figure 3, while typological integration (+T_D) improves over the baseline, it is significantly outperformed by simple fine-tuning on a minimal amount of target-language data. Fine-tuning on just 10 sentences yielded a 2.3x larger increase in UAS compared to the best typology-based method.
Redundancy in Neural Encoders
Probing experiments revealed that standard BiLSTM encoders can predict typological features from POS sequences with over 90% accuracy. This suggests that much of the information provided by external typological databases is already captured implicitly by the model's internal representations, explaining the "mixed" results seen in previous literature.
Conclusion
Typological similarity is a useful but imperfect proxy for transferability in neural dependency parsing. Future research should focus on developing task-specific typological metrics that align more closely with corpus statistics and the specific needs of the parsing architecture, moving beyond static, hand-coded databases.
Summary updated