Inducing and Using Alignments for Transition-based AMR Parsing

Andrew Drozdov, Jiawei Zhou, Radu Florian, Andrew McCallum, Tahira Naseem, Yoon Kim, Ramón Fernandez Astudillo

Abstract

This paper simplifies the complex, rule-heavy pipeline of Abstract Meaning Representation (AMR) parsing by introducing a neural aligner that automatically learns the mapping between words and graph nodes. By treating these alignments as a probability distribution rather than a single "best guess," the authors achieve new state-of-the-art results for gold-only trained models, particularly excelling in generalizing to new datasets like AMR 3.0.

Introduction

Abstract Meaning Representation (AMR) parsing involves transforming natural language sentences into semantic graphs that represent "who is doing what to whom." Transition-based parsers are a popular method for this task, but they require an alignment—a "map" connecting specific words in a sentence to specific nodes in the graph.

Traditionally, these alignments are not provided in datasets and must be inferred through a complex series of rule-based scripts, lemmatizers, and post-processing steps. These pipelines are often brittle and fail to generalize when the text domain changes. Furthermore, most parsers rely on a single "best guess" alignment, ignoring the inherent ambiguity in language. This paper addresses these issues by proposing a neural aligner and training strategies that account for alignment uncertainty.

Methods

The authors introduce a two-part solution to improve AMR parsing through better alignment handling.

The Neural Aligner

Instead of relying on heuristics, the authors use a neural sequence-to-sequence model with "hard attention." This model learns latent alignments by marginalizing over all possible word-to-node connections. By incorporating ELMo character-aware embeddings, the aligner handles unseen words more effectively than traditional count-based models. To apply this seq2seq approach, the AMR graph is first linearized into a tree structure.

Overview of the AMR parsing components and alignment matrix.

Training with Uncertainty

The research moves beyond using only the most likely (MAP) alignment. Two key strategies are explored:

  • Posterior Regularization (PR): This encourages the parser’s internal alignment logic to match the neural aligner’s distribution. Training on multiple sampled alignments acts as a form of data augmentation, preventing the model from over-relying on a single, potentially flawed alignment.
  • Importance Sampling (IS): This technique reweights samples from the aligner to better approximate the true likelihood of the graph.

Evaluation and Results

The models were evaluated on the AMR 2.0 and AMR 3.0 benchmarks. AMR 3.0, being larger and more diverse, specifically tests the model's ability to generalize.

Parsing Performance

On the AMR 3.0 dataset, the proposed model achieved a Smatch score of 83.1, setting a new state-of-the-art for models trained only on gold data (without synthetic "silver" data). Notably, the model achieves these results using simple greedy decoding, matching the performance of more computationally expensive beam search methods.

Smatch scores on AMR 2.0 and AMR 3.0 benchmarks.

Alignment Quality

The neural aligner demonstrated superior quality, scoring 96.5 F1 against ground-truth alignments. This significantly outperforms the previous rule-based SB-Align pipeline (89.2 F1) and rivals specialized models like LEAMR (97.4 F1).

Conclusion

The transition from hand-crafted heuristics to a unified neural alignment model simplifies the AMR parsing pipeline while improving accuracy and robustness. The core insight is that treating alignment as a distribution rather than a single point of truth provides a powerful signal for training. While the linearization process currently imposes some structural constraints, the overall approach offers a more "plug-and-play" solution for semantic parsing. Future work will look into combining these techniques with large-scale "silver" data augmentation.

Summary updated

BibTeX
@inproceedings{drozdov2022inducing,
  author = {Andrew Drozdov and Jiawei Zhou and Radu Florian and Andrew McCallum and Tahira Naseem and Yoon Kim and Ramón Fernandez Astudillo},
  title = {{Inducing and Using Alignments for Transition-based AMR Parsing}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=qrKaGOuY_ZlU},
}
Figure