AMR Parsing as Graph Prediction with Latent Alignment

Chunchuan Lyu, Ivan Titov

Abstract

Abstract meaning representations (AMRs) represent sentences as rooted labeled directed acyclic graphs. AMR parsing is challenging due to the lack of annotated alignments between nodes in the graphs and words in the corresponding sentences. We introduce a neural parser which treats alignments as latent variables within a joint probabilistic model of concepts, relations, and alignments. Using the variational auto-encoding framework and a continuous relaxation of discrete alignments, we show that joint modeling is preferable to traditional pipeline approaches. The parser achieves a state-of-the-art Smatch score of 74.4% on the LDC2016E25 benchmark.

Introduction

Abstract Meaning Representation (AMR) describes the semantic structure of a sentence—the "who did what to whom"—using a directed graph. However, training models to produce these graphs is difficult because the training data is "unaligned": we have the sentence and the target graph, but no explicit mapping between specific words and specific nodes. Traditionally, this was addressed using a pipeline approach: a separate tool would guess alignments, and a parser would then be trained on those guesses. This method is prone to error propagation, as the aligner is unaware of the final parsing objective.

Methodology

This paper proposes a joint model that learns to align, identify concepts, and predict relations simultaneously. The architecture treats alignments as latent variables, essentially asking which alignment makes a specific graph most likely given a sentence.

The neural architecture for joint alignment and parsing, featuring an RNN encoder and a classifier for relation prediction.

Key technical innovations include:

  • Variational Autoencoding (VAE): Used to maximize a lower bound of the likelihood, as marginalizing over all possible alignments is computationally infeasible.
  • Gumbel-Sinkhorn Relaxation: To make discrete alignments differentiable, the authors use a continuous relaxation that mimics a permutation matrix, allowing for end-to-end gradient descent.
  • Injective Constraints: Unlike standard attention mechanisms, this model enforces that each node aligns to a specific word, forcing the model to be more decisive.
  • Multi-pass Encoding: Separate BiLSTM encoders are utilized for identifying concepts and relations, allowing specialized internal representations for each task.

Experimental Results

The model was evaluated on two standard benchmarks: LDC2016E25 (R2) and LDC2015E86 (R1). The joint induction approach significantly outperformed previous models across various metrics.

Comparison of Smatch scores on R1 and R2 datasets against previous state-of-the-art models.

As shown in the results, the parser achieved a 74.4% Smatch score on the R2 dataset, representing a 3.4% absolute improvement over the previous best result. Ablation studies confirmed that replacing latent alignment with a standard pre-trained aligner (JAMR) caused the Smatch score to drop by 0.9%, validating the benefit of joint induction.

Key Insights

  • Joint Training Synergy: The parsing objective actively helps the aligner. When the model predicts a relationship between two nodes, it receives a signal that helps refine the alignment of those nodes within the sentence.
  • Resolving Ambiguity: In sentences with repeated words (e.g., "long hours and lots of long nights"), a concept-only model might struggle with mapping. However, the joint relation model identifies that one "long" modifies "hours" and the other modifies "nights," guiding the alignment model to resolve the ambiguity correctly.
  • Exploration via Soft Loss: Utilizing a "softer" version of the loss function (an alpha-norm) encouraged the model to explore different alignment possibilities early in training, preventing it from settling into local minima.

Conclusion and Future Work

While the joint model sets a new benchmark, it still relies on some rule-based preprocessing for "recategorization" (grouping subgraphs like dates) and simple frequency-based lookups for post-processing tasks like Wikification. Future work may involve replacing BiLSTMs with Graph Convolutional Networks (GCNs) to better capture the structural neighborhood of the AMR graph, potentially further increasing accuracy.

Summary updated

Abstract

Abstract meaning representations (AMRs) are broad-coverage sentence-level semantic representations. AMRs represent sentences as rooted labeled directed acyclic graphs. AMR parsing is challenging partly due to the lack of annotated alignments between nodes in the graphs and words in the corresponding sentences. We introduce a neural parser which treats alignments as latent variables within a joint probabilistic model of concepts, relations and alignments. As exact inference requires marginalizing over alignments and is infeasible, we use the variational auto-encoding framework and a continuous relaxation of the discrete alignments. We show that joint modeling is preferable to using a pipeline of align and parse. The parser achieves the best reported results on the standard benchmark (74.4% on LDC2016E25).
BibTeX
@inproceedings{lyu2018amr,
  author = {Chunchuan Lyu and Ivan Titov},
  title = {{AMR Parsing as Graph Prediction with Latent Alignment}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2018},
  url = {https://openreview.net/forum?id=HyW8O3g_bH},
}

Loading related content...

Figure