Abstract
We study multi-source transfer parsing for resource-poor target languages; specifically methods for target language adaptation of delexicalized discriminative graph-based dependency parsers. We first show how recent insights on selective parameter sharing, based on typological and language-family features, can be applied to a discriminative parser by carefully decomposing its model features. We then show how the parser can be relexicalized and adapted using unlabeled target language data and a learning method that can incorporate diverse knowledge sources through ambiguous labelings. In the latter scenario, we exploit two sources of knowledge: arc marginals derived from the base parser in a self-training algorithm, and arc predictions from multiple transfer parsers in an ensemble-training algorithm. Our final model outperforms the state of the art in multi-source transfer parsing on 15 out of 16 evaluated languages.
Introduction
Syntactic dependency parsing is a cornerstone of natural language processing, providing the structural foundation for tasks like machine translation and information extraction. However, state-of-the-art parsers typically require large, manually annotated treebanks, which are unavailable for the vast majority of the world's languages. Cross-lingual transfer addresses this by training parsers on resource-rich source languages and applying them to a target language.
While generative models have traditionally dominated this field, they often rely on restrictive independence assumptions. Discriminative models, such as graph-based MSTParsers, offer higher accuracy in supervised settings but struggle in zero-shot transfer because their rich features often overfit to source-specific word orders. This paper introduces a framework to adapt these powerful discriminative parsers to low-resource languages.
Methodology
The authors bridge the gap between discriminative parsing and cross-lingual transfer through two primary innovations: feature decomposition and ambiguity-aware training.
Feature-Based Selective Sharing
To mitigate overfitting, the authors decompose the features of a standard discriminative parser. They define a "Bare" model that captures universal selectional preferences (e.g., adjectives modifying nouns) while stripping out directional and word-order features. These features are then selectively reintroduced only when the source and target languages share specific typological traits—identified via the WALS database—or belong to the same language family.
Ambiguity-Aware Training
To adapt the parser using unlabeled target data, the authors move beyond standard self-training, which can reinforce a model's errors. Instead, they employ Ambiguity-Aware Self-Training (AAST). This method extracts a set of highly probable dependency arcs based on the base parser's marginal probabilities. The model is then trained to maximize the likelihood of this "ambiguous labeling," allowing it to use target-language lexical correlations to resolve syntactic uncertainty. This is further extended into Ambiguity-Aware Ensemble-Training (AAET), which integrates predictions from multiple transfer parsers.
Results and Discussion
The proposed methods were evaluated on CoNLL treebanks across 16 languages. The results demonstrate a clear progression in performance:
- Baseline Generative Model: 59.3% UAS
- Discriminative with Selective Sharing: 62.0% UAS
- AAST (Self-Training): 64.0% UAS
- AAET (Ensemble-Training): 65.4% UAS
The final AAET model achieved state-of-the-art results on 15 out of 16 languages, representing a 13% relative error reduction over previous methods. A key finding is that the primary benefit of self-training is "relexicalization"—the process by which a delexicalized (POS-only) parser learns word-specific features in the target language.
Conclusion
This work demonstrates that discriminative parsers can be effectively adapted for cross-lingual transfer. While a gap still exists between transfer performance and fully supervised models, the high "oracle" accuracy of the ambiguous labelings (75.7%) suggests significant room for further improvement. Future work will explore more sophisticated training algorithms to close this gap and further enhance parsing for resource-poor languages.
Summary updated