Introduction
Dependency parsing is a fundamental task in Natural Language Processing (NLP) that identifies the grammatical structure of a sentence by determining the relationships between words. For example, in the sentence "He has good control," a parser identifies "has" as the root verb, "He" as its subject, and "control" as its direct object.
While high-quality parsers exist for resource-rich languages like English, most of the world's 7,000+ languages lack the large-scale, manually annotated datasets (treebanks) required for modern machine learning. This paper addresses this "low-resource" challenge by proposing a framework for cross-lingual transfer parsing.
The Challenge of Cross-Lingual Transfer
Cross-lingual transfer involves training a model on a resource-rich source language and applying it to a resource-poor target language. Historically, this required "delexicalizing" the model—stripping away actual words and relying only on universal Part-of-Speech (POS) tags (e.g., NOUN, VERB). While POS tags provide a common ground, ignoring the words themselves discards a massive amount of syntactic information. The authors bridge this "lexical gap" by mapping words from different languages into a shared, distributed vector space.
Methodology: Neural Transition-Based Parsing
The proposed framework utilizes a transition-based neural network parser. Unlike traditional linear models that rely on manually engineered discrete features, this neural architecture can automatically learn complex interactions between features.
As illustrated in Figure 3, the model processes both lexical features (words, clusters) and non-lexical features (POS tags, relations, distance/valency) through specialized lookup tables. A key innovation is the use of a cube activation function ($h = (W_1 \cdot x + b_1)^3$). The authors demonstrate that this cubic non-linearity is mathematically equivalent to a low-rank tensor model, allowing the network to capture third-order feature interactions efficiently.
To ensure that word embeddings are transferable across languages, the authors propose two main induction methods:
- Robust Projection: Using word alignments from parallel text to project source embeddings onto target words. To handle out-of-vocabulary (OOV) words, they use a morphology-based approach that propagates embeddings based on edit distance.
- Canonical Correlation Analysis (CCA): Mapping pre-trained monolingual embeddings into a common space using a bilingual dictionary.
Experimental Results and Adaptation
The model was evaluated using the Universal Dependency Treebanks (UDT) v2.0 across English, German, Spanish, and French. The proposed "PROJ+Cluster" model achieved a 10.9% relative error reduction in Labeled Attachment Score (LAS) compared to standard delexicalized baselines.
Furthermore, the paper explores Target-Language Adaptation. Recognizing that languages have unique syntactic quirks (e.g., word order differences), the authors introduced a "minimal supervision" phase. By fine-tuning the transferred model on as few as 100 annotated sentences in the target language, they observed a massive jump in accuracy. For German, the Unlabeled Attachment Score (UAS) increased from 60.35% to 68.91%, nearly matching the performance of a model trained on 1,000 sentences from scratch.
Conclusion
This research demonstrates that distributed word representations are a powerful tool for cross-lingual transfer. By treating words as vectors in a shared space rather than discrete symbols, the model can leverage lexical knowledge across language boundaries, significantly improving parsing performance for low-resource languages.
Summary updated