SJTU at MRP 2019: A Transition-Based Multi-Task Parser for Cross-Framework Meaning Representation Parsing

Hongxiao Bai, Hai Zhao2019 · DOI

Abstract

This paper presents the SJTU system for the CoNLL 2019 Shared Task on Cross-Framework Meaning Representation Parsing (MRP). We propose a unified transition-based parser that handles five distinct semantic frameworks (DM, PSD, EDS, UCCA, and AMR) using a universal set of transition actions. Our multi-task architecture leverages shared BiLSTM layers to capture common linguistic features while using task-specific layers for framework nuances. The system achieved a 42% $F_1$ unified MRP score, demonstrating the effectiveness of a single model in learning across diverse graph-structured representations.

Introduction

Meaning Representation Parsing (MRP) is a critical task in Natural Language Processing that aims to map natural language sentences into formal semantic graphs. These graphs represent "who did what to whom," capturing the underlying meaning. However, the field is currently fragmented into multiple frameworks with varying structural properties:

  • DM (DELPH-IN MRS) and PSD (Prague Semantic Dependencies): Bi-lexical dependencies where nodes correspond directly to surface tokens.
  • EDS (Elementary Dependency Structures) and UCCA (Universal Conceptual Cognitive Annotation): Nodes can anchor to arbitrary substrings or multiple tokens.
  • AMR (Abstract Meaning Representation): Highly abstract representations where nodes may have no direct anchoring to the source text.

Developing separate parsers for each framework is inefficient and ignores the underlying commonalities of human language. The SJTU team addresses this by creating a single, cross-framework parser capable of handling anchored and unanchored, as well as rooted and unrooted structures, within a unified multi-task learning framework.

Methodology

The core of our approach is a transition-based parsing system that constructs semantic graphs incrementally using a stack-and-buffer mechanism.

Universal Transition Set

To accommodate the diverse requirements of the five frameworks, we defined a set of eight core transition actions. These include standard actions like Shift, Reduce, Left-Arc, Right-Arc, and Swap, alongside framework-specific actions such as Drop (to exclude non-semantic words), New(c) (for abstract node generation in EDS/AMR), and Add (for non-terminal nodes in UCCA).

The universal transition action set for cross-framework parsing.

Multi-Task Neural Architecture

Our model employs a hybrid BiLSTM architecture designed for multi-task learning. It consists of a shared BiLSTM to capture general linguistic features common across all five datasets and task-specific BiLSTMs to model the unique structural nuances of individual frameworks. The input layer combines pre-trained GloVe and BERT embeddings with linguistic features such as POS tags, lemmas, and syntactic dependency labels.

Neural architecture of the SJTU multi-task parser.

Training and Inference

We utilized a dynamic oracle during training to improve the model's robustness to sub-optimal states. Inference is performed via greedy decoding, followed by a framework-specific pruning step to ensure the generated graphs adhere to the target framework's constraints.

Datasets

The system was evaluated on the CoNLL 2019 Shared Task (MRP) data, covering DM, PSD, EDS, UCCA, and AMR. Additionally, the "The Little Prince" (lpps) dataset provided a cross-framework evaluation set of 100 annotated sentences from the famous novella.

Results and Discussion

Our system achieved a unified MRP $F_1$ score of 42% in the official evaluation. Key findings include:

  • Framework Performance: The model performed exceptionally well on AMR, achieving an $F_1$ of 37.3, significantly higher than the TUPA baseline (32.8).
  • Multi-Task Synergy: The shared architecture allowed data-rich frameworks (like AMR or DM) to bolster performance on data-scarce ones by providing better underlying representations.
  • Precision-Recall Trade-off: The parser demonstrated high precision but lower recall across the board. For instance, on EDS, it reached 75.0 precision (vs. TUPA's 74.0) but only 41.0 recall. This suggests the model tends to produce conservative, "smaller" graphs.

Conclusion

The SJTU transition-based multi-task parser demonstrates that semantic parsing across disparate frameworks can be unified into a single sequence-prediction problem. While the model achieves high precision, future work will focus on closing the recall gap by optimizing the transition set and the training oracle to better capture the density of semantic graphs.

Summary updated

Abstract

This paper describes the system of our team SJTU for our participation in the CoNLL 2019 Shared Task: Cross-Framework Meaning Representation Parsing. The goal of the task is to advance data-driven parsing into graphstructured representations of sentence meaning. This task includes five meaning representation frameworks: DM, PSD, EDS, UCCA, and AMR. These frameworks have different properties and structures. To tackle all the frameworks in one model, it is needed to find out the commonality of them. In our work, we define a set of the transition actions to oncefor-all tackle all the frameworks and train a transition-based model to parse the meaning representation. The adopted multi-task model also can allow learning for one framework to benefit the others. In the final official evaluation of the shared task, our system achieves 42% F 1 unified MRP metric score.
BibTeX
@misc{bai2019sjtu,
  author = {Hongxiao Bai and Hai Zhao},
  title = {{SJTU at MRP 2019: A Transition-Based Multi-Task Parser for Cross-Framework Meaning Representation Parsing}},
  year = {2019},
  doi = {10.18653/v1/k19-2008},
}
Figure