Inducing and Using Alignments for Transition-based AMR Parsing

Andrew Drozdov, Jiawei Zhou, Radu Florian, Andrew McCallum, Tahira Naseem, Yoon Kim, Ramón Fernandez Astudillo

Abstract: Transition-based parsers for Abstract Meaning Representation (AMR) rely on node-to-word alignments that are typically learned separately through complex, rule-based pipelines. These pipelines are often brittle and provide only a point-estimate of alignments, ignoring the inherent ambiguity in semantic mapping. This paper introduces a neural aligner that learns alignments without domain-specific rules and explores a tighter integration of aligner and parser training. By considering a distribution over oracle action sequences arising from aligner uncertainty, the proposed approach achieves state-of-the-art performance on AMR 3.0, matching silver-trained models using only gold data and efficient greedy decoding.

Introduction

Abstract Meaning Representation (AMR) encodes the semantic structure of a sentence as a rooted, directed graph where nodes represent concepts and edges represent relations. Transition-based parsing is a popular approach for AMR, where a parser constructs the graph through a sequence of actions. However, training these parsers requires "alignments"—mappings between the words in a sentence and the nodes in the resulting graph.

Historically, these alignments have been generated by external, rule-based pipelines such as SB-Align. These systems are often brittle, relying on lemmatizers and hand-coded heuristics that fail to generalize across different text domains. Furthermore, they provide a single "best guess" alignment, forcing the parser to treat potentially ambiguous mappings as absolute truth. This work addresses these limitations by inducing alignments via a neural model and utilizing the resulting probability distribution to improve parser robustness.

The Neural Aligner

To move away from hand-crafted rules, the authors propose a neural aligner based on a sequence-to-sequence architecture with hard attention. The model consists of:

  • Encoder: A Bi-LSTM that processes the input sentence, enhanced by pretrained ELMo character-aware embeddings. These embeddings are critical for capturing context and handling out-of-vocabulary terms.
  • Decoder: A uni-directional LSTM that generates a linearized version of the AMR graph.
  • Latent Variable Modeling: The aligner treats the word-to-node mapping as a latent variable. During training, the model marginalizes over all possible alignments, learning to associate nodes with words in an unsupervised manner without requiring manual alignment labels.

This approach is domain-agnostic and significantly more flexible than traditional pipelines, as it does not rely on language-specific preprocessing like part-of-speech tagging or lemmatization.

Integrating Aligner Uncertainty

Rather than simply taking the most likely alignment (the MAP estimate) to train the parser, this research explores training on the distribution of alignments. This is achieved through two primary methods:

  1. Posterior Regularization (PR): The parser is trained on multiple alignment samples drawn from the aligner’s distribution. This introduces the aligner's uncertainty as an inductive bias, effectively regularizing the parser and making it more robust to the "noise" inherent in semantic mapping.
  2. Importance Sampling (IS): The aligner acts as a surrogate model to approximate the true likelihood of the graph given the sentence, providing a more mathematically rigorous way to handle the latent alignment variable.

Experimental Results

The methodology was evaluated on the AMR 2.0 and AMR 3.0 benchmarks. AMR 3.0 is particularly challenging as it includes diverse genres such as Wikipedia and LORELEI data.

Alignment Quality

Intrinsic evaluation on a gold-aligned test set showed that the neural aligner achieved a 96.5 F1 score. A key finding was the impact of pretraining; without ELMo embeddings, the F1 score dropped to approximately 80, demonstrating that contextual word representations are essential for high-quality alignment induction.

Parsing Performance

The parser achieved a Smatch score of 83.1 on AMR 3.0. This is a significant result for several reasons:

  • Gold-Only Efficiency: The model reached state-of-the-art levels using only gold-standard training data, matching the performance of previous systems that required massive amounts of synthetic "silver" data.
  • Decoding Speed: Unlike many top-performing parsers that require computationally expensive beam search, this model achieves its results using simple greedy decoding, making it much faster for real-world applications.
  • Generalization: The neural approach proved far more robust than rule-based pipelines when moving from AMR 2.0 to the more diverse AMR 3.0 dataset.

Conclusion

By treating alignment as a probabilistic latent variable rather than a fixed preprocessing step, this work demonstrates that transition-based AMR parsers can become both more accurate and more efficient. The transition from brittle, rule-based pipelines to uncertainty-aware neural models allows for better generalization across domains and reduces the reliance on synthetic data. Future work may explore hybrid training strategies that combine the strengths of Posterior Regularization and Importance Sampling to further refine the integration between aligners and parsers.

Summary updated

Abstract

Transition-based parsers for Abstract Meaning Representation (AMR) rely on node-to-word alignments. These alignments are learned separately from parser training and require a complex pipeline of rule-based components, pre-processing, and post-processing to satisfy domain-specific constraints. Parsers also train on a point-estimate of the alignment pipeline, neglecting the uncertainty due to the inherent ambiguity of alignment. In this work we explore two avenues for overcoming these limitations. First, we propose a neural aligner for AMR that learns node-to-word alignments without relying on complex pipelines. We subsequently explore a tighter integration of aligner and parser training by considering a distribution over oracle action sequences arising from aligner uncertainty. Empirical results show this approach leads to more accurate alignments and generalization better from the AMR2.0 to AMR3.0 corpora. We attain a new state-of-the art for gold-only trained models, matching silver-trained performance without the need for beam search on AMR3.0.
BibTeX
@inproceedings{drozdov2022inducing,
  author = {Andrew Drozdov and Jiawei Zhou and Radu Florian and Andrew McCallum and Tahira Naseem and Yoon Kim and Ramón Fernandez Astudillo},
  title = {{Inducing and Using Alignments for Transition-based AMR Parsing}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=qrKaGOuY_ZlU},
}
Figure