Abstract
This paper describes the system of our team Phoenix for participating in the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Given the annotated gold standard data in CoNLL-U format, we train the tokenizer, tagger, and parser separately for each treebank based on the open-source pipeline tool UDPipe. Our system reads plain text input, performs preprocessing steps (tokenization, lemmas, morphology), and finally outputs syntactic dependencies. For low-resource languages with no training data, we use cross-lingual techniques to build models using related languages. In the official evaluation, our system achieves macro-averaged scores of 65.61%, 52.26%, and 55.71% for LAS, MLAS, and BLEX, respectively.
Introduction
Dependency parsing—the task of mapping grammatical structures within sentences—is a cornerstone of natural language processing. However, real-world applications often face two significant challenges: processing raw, unsegmented text and handling languages with limited or no annotated training data. The CoNLL 2018 Shared Task formalized these hurdles, requiring systems to perform end-to-end parsing from raw strings across a diverse set of 57 languages. Team Phoenix (Shanghai Jiao Tong University) addressed these challenges by developing a modular pipeline with a specific focus on cross-lingual enhancement for low-resource scenarios.
System Architecture
The Phoenix system is built upon the modular UDPipe 1.2 framework, customized into three distinct stages to transform raw text into syntactic dependency trees.
- Tokenization & Segmentation: The system employs a single-layer bidirectional Gated Recurrent Unit (Bi-GRU) network. This model operates at the character level to predict token and sentence boundaries, a method that proved highly effective in official evaluations.
- Tagging & Lemmatization: Utilizing the MorphoDita tool, the pipeline generates Part-of-Speech (POS) tags and lemmas. This stage combines a morphological guesser with an averaged perceptron to handle word-form variations.
- Parsing: The final syntactic structure is produced by Parsito, a transition-based neural network parser that identifies word heads and dependency relations.
Low-Resource Language Enhancement
A core contribution of this work is the "mixture model" strategy for languages with zero training data. For these low-resource cases, the authors manually curated training sets by merging data from linguistically related "neighbor" languages. For instance, a model for Breton was trained on a combination of Irish, Latin, Old French, and French. This approach leverages shared grammatical features to bridge the data gap, allowing the system to produce meaningful parses where standard models would fail.
Experimental Results
The system was evaluated on the Universal Dependencies (UD) version 2.2 treebanks, covering 82 test sets. Phoenix achieved competitive macro-averaged scores: 65.61% LAS, 52.26% MLAS, and 55.71% BLEX.
A standout performance was noted in sentence segmentation, where the system ranked 1st for the Parallel Universal Dependencies (PUD) treebanks. In the low-resource category, the mixture model approach yielded a macro-averaged LAS of 16.99%.
While the performance on low-resource languages remains significantly lower than on "Big" treebanks (which averaged 73.93% LAS), the results demonstrate that cross-lingual transfer is a viable path forward for zero-data scenarios.
Conclusion
The Phoenix system demonstrates the effectiveness of a modular neural pipeline for end-to-end multilingual parsing. The results highlight two key insights: character-level Bi-GRUs are robust for segmentation, and cross-lingual "mixture models" provide a necessary baseline for low-resource languages. Future work will focus on automating the selection of linguistic neighbors and mitigating the noise introduced when merging disparate treebanks.
Summary updated