Saarland at MRP 2019: Compositional parsing across all graphbanks

Lucia Donatelli, Meaghan Fowlie, Jonas Groschwitz, Alexander Koller, Matthias Lindemann, Mario Mina, Pia WeißenhornCoNLL Shared Task 2019 · OpenReview

Abstract: Lucia Donatelli, Meaghan Fowlie, Jonas Groschwitz, Alexander Koller, Matthias Lindemann, Mario Mina, Pia Weißenhorn. Proceedings of the Shared Task on Cross-Framework Meaning Representation Parsing at the 2019 Conference on Natural Language Learning.

2019.

Introduction

Semantic parsing—the task of translating natural language into machine-readable meaning representations—has long been a fragmented field. Different linguistic theories have produced a variety of "graphbanks," such as Abstract Meaning Representation (AMR), UCCA, DM, PSD, and EDS. Each of these frameworks has its own structural idiosyncrasies, ranging from bi-lexical dependencies to hierarchical phrase-like structures.

The 2019 Cross-Framework Meaning Representation Parsing (MRP) shared task challenged the community to move beyond specialized, framework-specific architectures and build a "universal" parser. The Saarland University team addressed this by developing a unified compositional framework based on the Apply-Modify (AM) algebra, demonstrating that a single mathematical abstraction can bridge the gaps between five structurally distinct semantic graph formats.

The AM Algebra

The core innovation of the Saarland system is the use of the AM algebra to decompose graph parsing into a more manageable dependency parsing task. Instead of predicting a complex, global graph structure in one step, the system works with small, local graph fragments called as-graphs or supertags.

Elementary graph fragments (as-graphs) representing semantic roles and concepts.

As shown in Figure 1, these fragments represent the semantic contribution of individual words. The AM algebra provides a formal system for combining these fragments using two primary operations:

  • Apply (APP): This operation fills a semantic role. For example, a subject fills a verb's "ARG0" slot.
  • Modify (MOD): This operation adds an optional descriptor, such as an adjective modifying a noun or an adverb modifying a verb.

AM Dependency Parsing

The team’s pipeline transforms the semantic graph into an intermediate AM dependency tree. This tree structure captures how the supertags for each word in a sentence are combined via algebraic operations to form the final meaning representation.

Representation of a sentence as an AM algebraic term and its corresponding dependency tree.

The neural architecture consists of several key components:

  1. Supertagger: A BiLSTM-based model that predicts the most likely graph fragment (supertag) for each word in the input sentence.
  2. Dependency Parser: A neural parser that predicts the edges between these fragments, effectively determining the "who does what to whom" relationships.
  3. Well-Typed Decoding: Because the AM algebra has strict formal rules, the team implemented a "fixed-tree decoder." This ensures that the predicted dependency tree is mathematically valid (well-typed) before it is converted back into the target graph format (e.g., AMR or DM).

For frameworks like UCCA, which use hierarchical nodes rather than simple word-to-word edges, the authors introduced preprocessing steps like "C-edge contraction" and "edge raising." These techniques flatten complex hierarchies into a format that the dependency parser can process, redistributing the graph's complexity into the supertags.

Results and Key Insights

The Saarland system proved highly competitive across the diverse graphbanks provided in the MRP 2019 task. Notably, they achieved the highest accuracy of all participants on the PSD framework with a 91.3 F-score. They also maintained strong performance on DM (94.7) and EDS (89.1).

While initial scores for AMR and UCCA were lower due to technical bugs and resource limitations (such as being restricted from using WordNet), post-deadline improvements demonstrated the system's potential, bringing the AMR F-score to 71.6 and UCCA to 70.1.

A key insight from the evaluation was the "dependency bottleneck." The authors found that for some frameworks (like PSD), performance was primarily limited by the supertagger's ability to pick the correct fragment. For others, like UCCA, the challenge lay in the dependency parser's ability to correctly connect those fragments. This suggests that the difficulty of semantic parsing is distributed differently across linguistic frameworks.

Conclusion and Future Work

The Saarland team's success at MRP 2019 underscores the power of mathematical abstraction in natural language processing. By treating graph parsing as a dependency task over a formal algebra, they created a highly portable system that doesn't need to "know" the specific rules of any one framework.

However, the system currently relies on handwritten heuristics to decompose training graphs into AM trees. This dependency on manual rules can lead to data loss if a graph cannot be decomposed—a significant issue for UCCA, where initial decomposition rates were low. Future work will focus on developing more flexible variants of the AM algebra and reducing the reliance on manual heuristics to further improve the parser's universality and robustness.

Summary updated

BibTeX
@inproceedings{donatelli2019saarland,
  author = {Lucia Donatelli and Meaghan Fowlie and Jonas Groschwitz and Alexander Koller and Matthias Lindemann and Mario Mina and Pia Weißenhorn},
  title = {{Saarland at MRP 2019: Compositional parsing across all graphbanks}},
  booktitle = {CoNLL Shared Task},
  year = {2019},
  url = {https://openreview.net/forum?id=qwF-sokG2jO},
}
Figure