Abstract
Abstract meaning representations (AMRs) are broad-coverage sentence-level semantic representations. AMRs represent sentences as rooted labeled directed acyclic graphs. AMR parsing is challenging partly due to the lack of annotated alignments between nodes in the graphs and words in the corresponding sentences. We introduce a neural parser which treats alignments as latent variables within a joint probabilistic model of concepts, relations and alignments. As exact inference requires marginalizing over alignments and is infeasible, we use the variational auto-encoding framework and a continuous relaxation of the discrete alignments. We show that joint modeling is preferable to using a pipeline of align and parse. The parser achieves the best reported results on the standard benchmark (74.4% on LDC2016E25).
Introduction
Abstract Meaning Representation (AMR) captures the semantic structure of a sentence, identifying "who is doing what to whom" in a rooted, directed acyclic graph. Unlike syntactic trees, AMR nodes (concepts) do not have a fixed one-to-one mapping to the surface words of a sentence. This lack of explicit alignment in standard datasets creates a significant hurdle for training supervised parsers.
Traditionally, researchers have relied on a "pipeline" approach: first using an external aligner to predict word-to-node mappings, and then training a parser on these fixed alignments. However, errors in the initial alignment phase propagate through the system, and the aligner remains unaware of the downstream parsing task. This paper addresses this "missing link" by treating alignments as latent variables.
Methodology: Latent Alignment via Gumbel-Sinkhorn
The core innovation of this work is a joint probabilistic model that handles concepts, relations, and alignments simultaneously. By modeling these elements together, the parser can "discover" the most linguistically useful alignments during training rather than relying on pre-computed guesses.
To overcome the non-differentiable nature of discrete alignments (where Word A maps to Node B), the authors employ two key techniques:
- Variational Autoencoding (VAE): Alignments are treated as latent variables, with an inference network approximating the posterior distribution.
- Gumbel-Sinkhorn Relaxation: This provides a continuous, differentiable approximation of permutation matrices. It allows the model to maintain "soft" alignments during training that gradually sharpen into hard, injective (one-to-one) assignments, enabling the use of standard backpropagation.
Model Architecture
The parser utilizes a BiLSTM encoder to generate contextualized word representations. These representations feed into two main components: a concept identifier and a relation classifier.
The relation identification stage uses a bi-affine classifier—a technique proven effective in dependency parsing—to predict edges and labels between concepts. To improve accuracy, the model employs a multi-pass encoding strategy, re-encoding the sentence for each identified predicate to focus the LSTM's attention on the relevant context for that specific part of the graph.
Experimental Results
The model was evaluated on the LDC2016E25 and LDC2015E86 benchmarks. By allowing the model to learn alignments that optimize the parsing objective, the joint approach significantly outperformed previous pipeline-based and sequence-to-sequence models.
The parser achieved a Smatch score of 74.4% on the LDC2016E25 test set, representing a 3.4% absolute improvement over the previous state-of-the-art. Analysis of subtask metrics revealed that the joint model is particularly effective at handling Reentrancy (nodes with multiple parents) and Semantic Role Labeling (SRL), suggesting that superior alignments lead to a more robust structural understanding of complex sentences.
Conclusion
Treating alignment as a latent variable within a joint probabilistic framework resolves a fundamental bottleneck in AMR parsing. The use of Gumbel-Sinkhorn relaxation allows for end-to-end training of discrete structures, proving that joint modeling is superior to traditional pipeline methods. This approach not only sets a new performance benchmark but also provides a more theoretically sound way to handle the structural ambiguity inherent in semantic representation.
Summary updated