Abstract
Jiawei Zhou, Tahira Naseem, Ramón Fernandez Astudillo, Young-Suk Lee, Radu Florian, Salim Roukos. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.
2021.
Introduction
Abstract Meaning Representation (AMR) parsing is the task of translating natural language sentences into semantic graphs where nodes represent concepts and edges represent their relationships. Recently, the field has shifted toward treating this as a sequence-to-sequence translation problem, linearizing graphs into strings for models like BART. While effective, this approach often fails to guarantee graph well-formedness and loses explicit word-to-node alignments.
Traditional transition-based parsers address these issues by constructing graphs step-by-step through specific actions, but they often rely on complex, hand-crafted rules. This paper introduces StructBART, a model that integrates a simplified transition system directly into the BART architecture to achieve the benefits of both generative power and structural constraints.
Methods
StructBART utilizes a streamlined transition set consisting of six basic actions: SHIFT (move to the next word), NODE (create a specific node), COPY (create a node from the current word), LA/RA (create Left/Right Arcs between nodes), and ROOT. By removing complex "subgraph" actions and re-categorization scripts, the model learns to handle entities and dates directly from the pre-trained data.
The architecture is modified to be "structure-aware":
- Pointer Network: A decoder self-attention head is repurposed to "point" back to previous nodes for edge creation (LA/RA actions).
- Hard Attention: Masking ensures the decoder focuses on the specific source word currently being processed by the "cursor."
- Vocabulary Strategies: The authors explored separate vocabularies for actions (
sep-voc) and a joint vocabulary shared with BART’s subword units (joint-voc).
Key Results
The models were evaluated on standard AMR benchmarks (1.0, 2.0, and 3.0), supplemented by "silver" data from PropBank and SQuAD 2.0. StructBART achieved a state-of-the-art 84.7 Smatch score on AMR 2.0, setting a new record at the time of publication.
Beyond raw performance, StructBART offers several critical advantages:
- Reliability: It guarantees 100% graph well-formedness as the transition system cannot physically create an invalid graph.
- Built-in Alignments: The model produces word-to-node alignments as a natural byproduct of the parsing process.
- Logical Robustness: It significantly outperforms pure linearization models on complex tasks like "Re-entrancy" (nodes with multiple parents) and Semantic Role Labeling (SRL).
Discussion and Limitations
A key insight of the study is that the pre-trained encoder's understanding of the source sentence is the primary driver of performance, contributing roughly 10 Smatch points. By simplifying the transition set, the model becomes more robust, relying on BART's internal knowledge rather than external scripts.
However, limitations remain. The model still requires an external post-processing step for Wikification. It also struggles with rare disconnected graphs and produces longer action sequences due to its granularity, which can impact decoding speed. Future work may involve applying these structure-aware principles to other large language models for diverse structured data tasks.
Summary updated