Abstract
Miryam de Lhoneux, Yan Shao, Ali Basirat, Eliyahu Kiperwasser, Sara Stymne, Yoav Goldberg, Joakim Nivre. Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.
2017.
The Uppsala University team demonstrates that high-performing dependency parsers can be constructed without the standard reliance on part-of-speech (POS) tags or morphological features. By employing a minimalist pipeline consisting of joint segmentation and a transition-based parser powered by character-level BiLSTMs, the team achieved top-tier results in sentence segmentation and a top-10 post-evaluation ranking in parsing accuracy at the CoNLL 2017 shared task.
The Challenge: Pipeline Error Propagation
In traditional Natural Language Processing (NLP), dependency parsing—mapping the grammatical structure of a sentence—is typically the final step in a sequential pipeline. This process usually requires segmenting text into sentences and words, assigning POS tags, and performing morphological analysis.
A significant drawback of this approach is that errors in early stages, such as incorrect POS tagging, propagate downward and degrade parsing quality. Furthermore, many low-resource languages lack the reliable taggers or analyzers necessary for this pipeline. This research explores whether a "bare-bones" system can move directly from raw text to syntactic trees by allowing deep learning models to learn linguistic features directly from character and word sequences.
Methodology: A Tag-less Architecture
The proposed system utilizes a two-stage pipeline designed for robustness and simplicity:
- Joint Segmentation: Instead of treating word and sentence boundaries as separate problems, the model uses a BiRNN-CRF (Bidirectional Recurrent Neural Network with a Conditional Random Field layer). This model predicts boundaries for words and sentences simultaneously from character inputs. It also identifies multiword tokens (e.g., Spanish del as de + el) using a dictionary or an encoder-decoder sequence-to-sequence network.
- Transition-Based Parsing: The core innovation is a greedy transition-based model (arc-hybrid) that eliminates the need for explicit POS tags.
- Character-level Representations: A BiLSTM creates word representations based on constituent characters, allowing the model to capture internal word structures (like suffixes) that typically signal POS categories.
- RSV Embeddings: Real-valued Syntactic Vectors are used to build a multilingual syntactic space based on universal feature distributions.
- Pseudo-Projective Parsing: To handle non-projective trees (common in languages with flexible word order), the system uses a pre-processing trick to flatten trees into a projective format for the greedy parser, which is later reconstructed.
Evaluation and Results
The system was evaluated on the CoNLL 2017 Shared Task dataset (Universal Dependencies v2.0), covering 81 treebanks across diverse language families, including "surprise" languages with minimal training data.
Key Findings
- Segmentation Excellence: The joint segmentation approach achieved an 89.03 F1 score for sentence segmentation, ranking 2nd overall in the competition.
- Parsing Accuracy: The official macro-averaged Labeled Attachment Score (LAS) F1 was 65.11. However, after resolving a serialization bug in the
dynetlibrary post-competition, the LAS improved to 70.49. This corrected score would have placed the team in the top 10 of 33 participants. - Morphological Robustness: The system significantly outperformed baselines on morphologically rich languages like Ancient Greek and Arabic, proving that character-based BiLSTMs can effectively "obviate" the need for explicit linguistic tags.
Discussion and Future Work
While the minimalist approach proved highly effective for standard and morphologically rich languages, it struggled with "zero-shot" scenarios in surprise languages where training data was nearly non-existent. The authors suggest that while tags are not strictly necessary for high performance, future hybrid models that jointly predict POS tags and morphology alongside the parse tree could further clarify the value of explicit linguistic information. Additionally, the team plans to explore training on larger, unannotated datasets to refine their multilingual word embeddings.
Summary updated