Abstract
This paper presents the system used in our submission to the IWPT 2020 Shared Task. Our system is a graph-based parser with second-order inference. For the low-resource Tamil corpus, we specially mixed the training data of Tamil with other languages and significantly improved the performance of Tamil. Due to our misunderstanding of the submission requirements, we submitted graphs that are not connected, which makes our system only rank 6th over 10 teams. However, after we fixed this problem, our system is 0.6 ELAS higher than the team that ranked 1st in the official results.
Introduction
Traditional dependency parsing typically results in a "tree" structure where every word has exactly one head. However, real-world language is more complex. Enhanced Universal Dependencies (EUD) go beyond trees to represent phenomena like shared subjects in control structures, cycles, and implied "empty nodes." This transition turns parsing into a graph prediction problem, presenting two primary challenges: structural complexity (reentrancies) and data scarcity for low-resource languages like Tamil, which provided only 400 training sentences.
Methodology
Our system approaches EUD parsing as a semantic dependency parsing (SDP) task, utilizing a three-stage architecture:
- The Encoder: We use a combination of pre-trained embeddings—XLMR, Flair, and FastText—fed into a three-layer BiLSTM to create rich, contextualized word representations.
- Second-Order Inference: Most parsers are "first-order," scoring potential edges independently. Our system uses Mean-Field Variational Inference (MFVI) to perform second-order inference, allowing the model to consider "sibling" and "grandparent" relationships between edges for more globally coherent graphs.
- Data Mixing (The Tamil Strategy): To address the Tamil data shortage, we mixed Tamil data with high-resource languages (like Czech). Crucially, we removed the labels from the high-resource data. This allowed the model to learn general linguistic structures from the large dataset while learning specific labels only from the Tamil data.
Experimental Results
The system was evaluated on the IWPT 2020 Shared Task dataset across 17 languages using the ELAS (Enhanced Labeled Attachment Score) metric.
The Connectivity Paradox
A misunderstanding of submission requirements led to our initial submission of non-connected graphs, resulting in an official 6th-place ranking. Post-evaluation revealed that while non-connected graphs can score slightly higher in raw metrics, applying the Maximum Spanning Tree (MST) algorithm to ensure connectivity allowed our system to outperform the official 1st-place team by 0.56 ELAS on average.
Low-Resource Performance
Our data-mixing strategy proved highly effective for Tamil. By combining Tamil data with Czech, we achieved a substantial boost in ELAS, demonstrating that structural transfer is possible even when label sets do not match.
Discussion and Conclusion
Our results demonstrate that second-order inference provides a significant performance advantage over first-order methods by considering edge interactions. Furthermore, the "Tamil Trick" shows that you don't need matching labels to help a low-resource language; stripping labels from a high-resource "donor" language still allows the model to learn valuable lessons about arc behavior.
While effective, our data-mixing approach currently relies on manual selection of donor languages. Future work will explore automated methods for selecting donor languages based on linguistic similarity and investigate the computational trade-offs of second-order inference in large-scale applications.
The code for this project is available at: https://github.com/Alibaba-NLP/MultilangStructureKD
Summary updated