TL;DR
This paper presents the LIGM-Alpage system for the SPMRL 2013 French dependency parsing task, achieving top results by combining multiple parsers in a "reparsing" architecture. Their key innovation lies in tackling the challenging scenario of jointly predicting both multi-word expressions (MWEs) and syntactic dependencies without relying on gold-standard annotations.
What Problem Does This Paper Address?
Identifying and parsing multi-word expressions (MWEs) – like "New York City" or "kick the bucket" – is a long-standing challenge in Natural Language Processing. While previous work often assumed MWEs were pre-identified (using "gold" annotations), this paper focuses on the more realistic scenario where MWEs must be identified automatically alongside syntactic dependencies. This is particularly difficult because errors in MWE identification can negatively impact parsing accuracy. The SPMRL 2013 shared task provided a unique opportunity to evaluate systems on this joint MWE and dependency parsing problem for French, especially in a setting where even basic morphological information wasn't provided as gold standard.
Methods
The LIGM-Alpage system employs a two-pronged approach:
- Pipeline Systems: MWEs are first identified and then merged into single tokens. A standard dependency parser is then applied to this modified sentence.
- Joint Systems: Dependency trees are generated directly, with MWEs implicitly encoded by specific dependency labels (e.g.,
dep_cpd). A post-processing step then assigns a Part-of-Speech (POS) tag to the MWEs.
The core of their winning system is a reparsing architecture. This involves taking the outputs from several different parsers (trained using both pipeline and joint approaches) and combining them to produce a final, improved parse. This is achieved by building a graph of all possible arcs from the individual parsers and then finding the maximum spanning tree within this graph. The weight of each arc is determined by its frequency across the input parses, and its label is the most voted label.
For MWE analysis, they use a Conditional Random Fields (CRF) model, enhanced with external lexicons. For parsing, they leverage three established parsers: MALT (a transition-based parser), Mate-tools 1 (a graph-based parser), and Mate-tools 2 (a joint POS tagger and transition-based parser).
Datasets
The primary dataset used is the French portion of the SPMRL 2013 Shared Task dataset. This dataset is notable for containing a significant number of MWEs and for evaluating systems in a "realistic" setting where gold-standard morphological information and MWE grouping are not provided. They also mention using external lexicons like Lefff and DELA to improve their system's performance, especially for out-of-vocabulary words.
Key Insights and Ideas
- Reparsing is Powerful: Combining the outputs of multiple parsers, even those with different architectures (pipeline vs. joint), significantly improves overall parsing accuracy. This suggests that different parsers capture complementary information.
- Joint MWE and Parsing is Challenging but Achievable: While automatic MWE identification can hurt parsing if done poorly, their system demonstrates that it's possible to achieve state-of-the-art results by carefully integrating MWE analysis with dependency parsing.
- External Lexicons are Crucial: The use of large-coverage French lexicons (Lefff, DELA) and specialized dictionaries (Prolex) significantly aids in both MWE identification and handling of unknown words.
- Pipeline MWE Analysis Outperforms Joint for MWE Recognition: Interestingly, while joint systems are competitive for overall parsing, a dedicated pipeline MWE analyzer performed better on MWE recognition tasks specifically.
Key Results
The LIGM-Alpage system achieved the best results on the French dependency parsing track of the SPMRL 2013 Shared Task, outperforming other participants across various metrics, including LAS (Labeled Attachment Score) and UAS (Unlabeled Attachment Score).
- Their best reparsing system achieved a LAS score of 86.23 on the development set (full training), outperforming the best single-parser systems.
- For MWE recognition (F-measure), their pipeline system achieved 81.7, and the best reparser achieved 81.8 on the development set.
- The system also performed well on MWE+POS recognition, achieving 77.9 F-measure.
Limitations and Future Work
- MWE Annotation Inconsistencies: The authors acknowledge that inconsistencies in the French Treebank's MWE annotations might explain some of the lower performance on open-class MWEs (like adjectives and verbs).
- Future Work: They plan to integrate features specific to MWEs directly into their joint parsing system. This, they hope, will allow the reparser to outperform both joint and pipeline systems not only on parsing but also on MWE recognition itself.
Summary updated