Abstract
This paper presents the SJTU system for the CoNLL 2019 Shared Task on Cross-Framework Meaning Representation Parsing (MRP). We propose a unified transition-based parser that handles five distinct semantic frameworks (DM, PSD, EDS, UCCA, and AMR) using a universal set of transition actions. Our multi-task architecture leverages shared BiLSTM layers to capture common linguistic features while using task-specific layers for framework nuances. The system achieved a 42% $F_1$ unified MRP score, demonstrating the effectiveness of a single model in learning across diverse graph-structured representations.
Introduction
Meaning Representation Parsing (MRP) is a critical task in Natural Language Processing that aims to map natural language sentences into formal semantic graphs. These graphs represent "who did what to whom," capturing the underlying meaning. However, the field is currently fragmented into multiple frameworks with varying structural properties:
- DM (DELPH-IN MRS) and PSD (Prague Semantic Dependencies): Bi-lexical dependencies where nodes correspond directly to surface tokens.
- EDS (Elementary Dependency Structures) and UCCA (Universal Conceptual Cognitive Annotation): Nodes can anchor to arbitrary substrings or multiple tokens.
- AMR (Abstract Meaning Representation): Highly abstract representations where nodes may have no direct anchoring to the source text.
Developing separate parsers for each framework is inefficient and ignores the underlying commonalities of human language. The SJTU team addresses this by creating a single, cross-framework parser capable of handling anchored and unanchored, as well as rooted and unrooted structures, within a unified multi-task learning framework.
Methodology
The core of our approach is a transition-based parsing system that constructs semantic graphs incrementally using a stack-and-buffer mechanism.
Universal Transition Set
To accommodate the diverse requirements of the five frameworks, we defined a set of eight core transition actions. These include standard actions like Shift, Reduce, Left-Arc, Right-Arc, and Swap, alongside framework-specific actions such as Drop (to exclude non-semantic words), New(c) (for abstract node generation in EDS/AMR), and Add (for non-terminal nodes in UCCA).
Multi-Task Neural Architecture
Our model employs a hybrid BiLSTM architecture designed for multi-task learning. It consists of a shared BiLSTM to capture general linguistic features common across all five datasets and task-specific BiLSTMs to model the unique structural nuances of individual frameworks. The input layer combines pre-trained GloVe and BERT embeddings with linguistic features such as POS tags, lemmas, and syntactic dependency labels.
Training and Inference
We utilized a dynamic oracle during training to improve the model's robustness to sub-optimal states. Inference is performed via greedy decoding, followed by a framework-specific pruning step to ensure the generated graphs adhere to the target framework's constraints.
Datasets
The system was evaluated on the CoNLL 2019 Shared Task (MRP) data, covering DM, PSD, EDS, UCCA, and AMR. Additionally, the "The Little Prince" (lpps) dataset provided a cross-framework evaluation set of 100 annotated sentences from the famous novella.
Results and Discussion
Our system achieved a unified MRP $F_1$ score of 42% in the official evaluation. Key findings include:
- Framework Performance: The model performed exceptionally well on AMR, achieving an $F_1$ of 37.3, significantly higher than the TUPA baseline (32.8).
- Multi-Task Synergy: The shared architecture allowed data-rich frameworks (like AMR or DM) to bolster performance on data-scarce ones by providing better underlying representations.
- Precision-Recall Trade-off: The parser demonstrated high precision but lower recall across the board. For instance, on EDS, it reached 75.0 precision (vs. TUPA's 74.0) but only 41.0 recall. This suggests the model tends to produce conservative, "smaller" graphs.
Conclusion
The SJTU transition-based multi-task parser demonstrates that semantic parsing across disparate frameworks can be unified into a single sequence-prediction problem. While the model achieves high precision, future work will focus on closing the recall gap by optimizing the transition set and the training oracle to better capture the density of semantic graphs.
Summary updated