Hitachi at MRP 2019: Unified Encoder-to-Biaffine Network for Cross-Framework Meaning Representation Parsing

Yuta Koreeda, Gaku Morio, Terufumi Morishita, Hiroaki Ozaki, Kohsuke Yanai

Abstract

This paper describes the proposed system of the Hitachi team for the Cross-Framework Meaning Representation Parsing (MRP 2019) shared task. In this shared task, the participating systems were asked to predict nodes, edges and their attributes for five frameworks, each with different order of "abstraction" from input tokens. We proposed a unified encoder-to-biaffine network for all five frameworks, which effectively incorporates a shared encoder to extract rich input features, decoder networks to generate anchorless nodes in UCCA and AMR, and biaffine networks to predict edges. Our system was ranked fifth with the macro-averaged MRP F1 score of 0.7604, and outperformed the baseline unified transition-based MRP. Furthermore, post-evaluation experiments showed that we can boost the performance of the proposed system by incorporating multi-task learning, whereas the baseline could not. These imply efficacy of incorporating the biaffine network to the shared architecture for MRP and that learning heterogeneous meaning representations at once can boost the system performance.

Introduction

Meaning Representation Parsing (MRP) involves transforming natural language sentences into semantic graphs. However, the field is fragmented across various frameworks such as AMR, UCCA, DM, PSD, and EDS, each with distinct rules for node and edge representation. Some frameworks align nodes directly to words, while others include "abstract" nodes representing concepts not explicitly present in the text. The Hitachi team addressed this challenge by developing a unified "Encoder-to-Biaffine" network capable of handling all five frameworks used in the CoNLL 2019 shared task, despite their varying levels of abstraction.

Methods

The proposed architecture consists of three primary stages designed to unify the parsing process across heterogeneous frameworks:

  1. Shared Encoder: To capture deep linguistic context, the system utilizes a "mega-embedding" approach, concatenating surface forms, lemmas, POS tags, and Named Entity labels with pre-trained embeddings like GloVe, ELMo, and BERT. These features are processed through a multi-layered bidirectional LSTM (biLSTM).
  2. Node Generation: The system handles the "missing node" problem differently depending on the framework. For UCCA, a Pointer Network identifies spans to generate non-terminal nodes. For AMR, an Extended Pointer-Generator Network handles concept nodes that do not have a 1:1 mapping to input tokens.
  3. Biaffine Edge Prediction: Once nodes are established, a Biaffine Classifier calculates the probability of edges and predicts their labels. This mechanism serves as a universal edge predictor across all graph types.
System Architecture

Additionally, the team explored Multi-task Learning (MTL), training the model on all five frameworks simultaneously to allow the shared encoder to learn robust, generalized semantic features.

Results

The model was evaluated on the MRP 2019 Shared Task datasets. The system achieved a macro-averaged F1 score of 0.7604, ranking 5th overall.

MRP 2019 Evaluation Results

Key performance highlights include:

  • Baseline Comparison: The system significantly outperformed the TUPA baseline (a unified transition-based parser) in four out of five frameworks.
  • Framework Strengths: The architecture performed exceptionally well on PSD (ranked 2nd) and UCCA (ranked 5th).
  • MTL Synergy: Post-competition experiments demonstrated that training on multiple frameworks simultaneously improved performance compared to single-framework models, suggesting that the model successfully exploits shared underlying semantic knowledge.
  • BERT Integration: While excluded from the formal competition due to time constraints, post-evaluation showed that BERT significantly boosted scores across the board.

Conclusion

The Hitachi team demonstrated that a biaffine architecture is flexible enough to serve as a universal edge predictor for diverse semantic graphs. While the system faced challenges with AMR's complex node generation, the overall results validate the efficacy of a unified encoder-to-biaffine approach. Future work will focus on developing a more elegant, universal node generation method to fully bridge the abstraction gap across all semantic representations.

Summary updated

Abstract

This paper describes the proposed system of the Hitachi team for the Cross-Framework Meaning Representation Parsing (MRP 2019) shared task. In this shared task, the participating systems were asked to predict nodes, edges and their attributes for five frameworks, each with different order of "abstraction" from input tokens. We proposed a unified encoder-to-biaffine network for all five frameworks, which effectively incorporates a shared encoder to extract rich input features, decoder networks to generate anchorless nodes in UCCA and AMR, and biaffine networks to predict edges. Our system was ranked fifth with the macro-averaged MRP F1 score of 0.7604, and outperformed the baseline unified transition-based MRP. Furthermore, post-evaluation experiments showed that we can boost the performance of the proposed system by incorporating multi-task learning, whereas the baseline could not. These imply efficacy of incorporating the biaffine network to the shared architecture for MRP and that learning heterogeneous meaning representations at once can boost the system performance.
BibTeX
@inproceedings{koreeda2019hitachi,
  author = {Yuta Koreeda and Gaku Morio and Terufumi Morishita and Hiroaki Ozaki and Kohsuke Yanai},
  title = {{Hitachi at MRP 2019: Unified Encoder-to-Biaffine Network for Cross-Framework Meaning Representation Parsing}},
  booktitle = {CoNLL Shared Task},
  year = {2019},
  url = {https://openreview.net/forum?id=GbD2dQhucl},
}
Figure