Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers

Adam Fisch, Jiang Guo, Regina Barzilay

Abstract

This paper explores the task of leveraging typology in the context of cross-lingual dependency parsing. While this linguistic information has shown great promise in pre-neural parsing, results for neural architectures have been mixed. Our investigation finds that: 1) the benefit of typological information is derived from coarsely grouping languages into syntactically-homogeneous clusters; 2) typology consistent with actual corpus statistics yields better transfer performance; and 3) typological similarity is only a rough proxy of cross-lingual transferability.

Introduction

Dependency parsing is a critical component of many NLP pipelines, yet it remains a challenge for the majority of the world's languages due to a lack of annotated treebanks. Cross-lingual transfer learning aims to solve this by training models on resource-rich languages and applying them to resource-poor ones. Linguistic typology—which categorizes languages based on structural features like word order—has traditionally been used to bridge the gap between source and target languages. However, in the era of neural parsers, the effectiveness of these features is less clear.

Methodology

We employ a state-of-the-art Biaffine Parser and investigate two primary methods for integrating typological information: Selective Sharing, where typology acts as a bias term, and Input Features, where typological vectors are concatenated to the encoder's input.

We evaluate three distinct typological representations:

  • $T_L$ (Linguistic Typology): Features derived from the World Atlas of Language Structures (WALS).
  • $T_D$ (Liu Directionalities): Vectors representing the average direction of dependency links within a corpus.
  • $T_S$ (Surface Statistics): Proxies for typology based on part-of-speech (POS) tag sequence frequencies.

Results and Analysis

The Clustering Effect

A key finding is that neural parsers utilize typological data primarily for coarse clustering rather than learning fine-grained compositional rules. As shown in Figure 1, languages naturally form clusters based on these features. Experiments replacing high-dimensional vectors with simple cluster IDs (K-Means) showed negligible performance loss, suggesting the model uses typology to identify "neighborhoods" of similar languages.

Figure 1: Visualization of language clusters based on typological features, illustrating the coarse grouping effect.

Performance Comparison

The results across 15 test languages indicate that $T_D$ (Liu Directionalities) provides the most consistent improvements. This is likely because $T_D$ reflects the actual statistical properties of the training corpora, whereas $T_L$ (WALS) represents idealized linguistic labels that may conflict with specific treebank distributions (e.g., the "Arabic Problem" where WALS labels Arabic as SV, but corpora are predominantly VS).

Figure 3: Main performance table showing UAS scores across different languages, methods, and typological representations.

As seen in Figure 3, while typological integration (+T_D) improves over the baseline, it is significantly outperformed by simple fine-tuning on a minimal amount of target-language data. Fine-tuning on just 10 sentences yielded a 2.3x larger increase in UAS compared to the best typology-based method.

Redundancy in Neural Encoders

Probing experiments revealed that standard BiLSTM encoders can predict typological features from POS sequences with over 90% accuracy. This suggests that much of the information provided by external typological databases is already captured implicitly by the model's internal representations, explaining the "mixed" results seen in previous literature.

Conclusion

Typological similarity is a useful but imperfect proxy for transferability in neural dependency parsing. Future research should focus on developing task-specific typological metrics that align more closely with corpus statistics and the specific needs of the parsing architecture, moving beyond static, hand-coded databases.

Summary updated

Abstract

This paper explores the task of leveraging typology in the context of cross-lingual dependency parsing. While this linguistic information has shown great promise in pre-neural parsing, results for neural architectures have been mixed. The aim of our investigation is to better understand this state-of-the-art. Our main findings are as follows: 1) The benefit of typological information is derived from coarsely grouping languages into syntactically-homogeneous clusters rather than from learning to leverage variations along individual typological dimensions in a compositional manner; 2) Typology consistent with the actual corpus statistics yields better transfer performance; 3) Typological similarity is only a rough proxy of cross-lingual transferability with respect to parsing.
BibTeX
@inproceedings{fisch2019working,
  author = {Adam Fisch and Jiang Guo and Regina Barzilay},
  title = {{Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=u73MTKCrZc},
}
Figure