Abstract
This paper explores the task of leveraging typology in the context of cross-lingual dependency parsing. While this linguistic information has shown great promise in pre-neural parsing, results for neural architectures have been mixed. The aim of our investigation is to better understand this state-of-the-art. Our main findings are as follows: 1) The benefit of typological information is derived from coarsely grouping languages into syntactically-homogeneous clusters rather than from learning to leverage variations along individual typological dimensions in a compositional manner; 2) Typology consistent with the actual corpus statistics yields better transfer performance; 3) Typological similarity is only a rough proxy of cross-lingual transferability with respect to parsing.
Introduction
In natural language processing, cross-lingual transfer aims to bridge the gap between resource-rich languages (e.g., English) and low-resource languages. For dependency parsing—the task of analyzing grammatical structure—linguistic typology offers a structured way to model language differences. Databases like the World Atlas of Language Structures (WALS) categorize languages by features such as word order (Subject-Verb-Object vs. Subject-Object-Verb). While traditional parsers benefited significantly from these features, modern neural architectures have shown inconsistent results. This study investigates the underlying reasons for this discrepancy, questioning whether neural models implicitly learn these rules or if the databases themselves are the bottleneck.
Methodology
To isolate syntactic effects, the study utilizes a delexicalized Deep Biaffine Attention parser, which operates solely on Part-of-Speech (POS) tags. Two primary methods for injecting typology are evaluated:
- Selective Sharing: A learned bias term is added to the parser's scoring function based on typological features.
- Input Features: Typology embeddings are concatenated directly to input POS tags.
The researchers compare three typological representations:
- Linguistic Typology ($T_L$): Hand-curated features from WALS.
- Liu Directionalities ($T_D$): Corpus-derived values representing the average direction of dependency relations.
- Surface Statistics ($T_S$): Proxy statistics derived from POS tag sequences.
Analysis: How Models Use Typology
A key finding is that neural models do not learn compositional rules from typology. Instead of interpreting features like "Subject-Verb" to adjust specific weights, the models use typology vectors as coordinates to cluster similar languages.
Replacing complex WALS features with simple K-Means cluster IDs resulted in negligible performance drops, confirming that the models primarily leverage typology for coarse grouping. Furthermore, the study found that BiLSTM encoders implicitly learn typological features from POS sequences; internal hidden states could predict WALS features with over 90% accuracy, rendering explicit inputs somewhat redundant.
Results and The "Reality Check"
Quantitatively, adding typology provides only modest gains. While the baseline parser achieved an average Unlabeled Attachment Score (UAS) of 67.53%, the best zero-shot typology method ($T_D$) only reached 69.57%.
Crucially, the study highlights a "reality check": fine-tuning the baseline model on just 10 annotated sentences in the target language achieved a UAS of 72.28%. This small amount of supervision outperformed the most complex zero-shot typology transfer methods by a factor of 2.3x. This suggests that current typological databases may be too noisy or mismatched with actual corpus statistics (e.g., WALS labeling Arabic as Subject-Verb while the corpus is predominantly Verb-Subject) to be fully effective in a zero-shot setting. Additionally, synthetic data via Galactic Dependencies (GD) provided a slight boost to weaker models but offered diminishing returns for the stronger baseline.
Conclusion
Integrating typology into neural dependency parsers remains challenging. The limited number of training languages hinders the learning of generalizable compositional rules, and typological similarity remains an imperfect proxy for parsing transferability. Future work should focus on aligning language similarity measures with actual transfer performance and developing architectures that can more effectively exploit the compositional nature of linguistic features.
Summary updated