Abstract: Transition-based parsers for Abstract Meaning Representation (AMR) rely on node-to-word alignments that are typically learned separately through complex, rule-based pipelines. These pipelines are often brittle and provide only a point-estimate of alignments, ignoring the inherent ambiguity in semantic mapping. This paper introduces a neural aligner that learns alignments without domain-specific rules and explores a tighter integration of aligner and parser training. By considering a distribution over oracle action sequences arising from aligner uncertainty, the proposed approach achieves state-of-the-art performance on AMR 3.0, matching silver-trained models using only gold data and efficient greedy decoding.
Introduction
Abstract Meaning Representation (AMR) encodes the semantic structure of a sentence as a rooted, directed graph where nodes represent concepts and edges represent relations. Transition-based parsing is a popular approach for AMR, where a parser constructs the graph through a sequence of actions. However, training these parsers requires "alignments"—mappings between the words in a sentence and the nodes in the resulting graph.
Historically, these alignments have been generated by external, rule-based pipelines such as SB-Align. These systems are often brittle, relying on lemmatizers and hand-coded heuristics that fail to generalize across different text domains. Furthermore, they provide a single "best guess" alignment, forcing the parser to treat potentially ambiguous mappings as absolute truth. This work addresses these limitations by inducing alignments via a neural model and utilizing the resulting probability distribution to improve parser robustness.
The Neural Aligner
To move away from hand-crafted rules, the authors propose a neural aligner based on a sequence-to-sequence architecture with hard attention. The model consists of:
- Encoder: A Bi-LSTM that processes the input sentence, enhanced by pretrained ELMo character-aware embeddings. These embeddings are critical for capturing context and handling out-of-vocabulary terms.
- Decoder: A uni-directional LSTM that generates a linearized version of the AMR graph.
- Latent Variable Modeling: The aligner treats the word-to-node mapping as a latent variable. During training, the model marginalizes over all possible alignments, learning to associate nodes with words in an unsupervised manner without requiring manual alignment labels.
This approach is domain-agnostic and significantly more flexible than traditional pipelines, as it does not rely on language-specific preprocessing like part-of-speech tagging or lemmatization.
Integrating Aligner Uncertainty
Rather than simply taking the most likely alignment (the MAP estimate) to train the parser, this research explores training on the distribution of alignments. This is achieved through two primary methods:
- Posterior Regularization (PR): The parser is trained on multiple alignment samples drawn from the aligner’s distribution. This introduces the aligner's uncertainty as an inductive bias, effectively regularizing the parser and making it more robust to the "noise" inherent in semantic mapping.
- Importance Sampling (IS): The aligner acts as a surrogate model to approximate the true likelihood of the graph given the sentence, providing a more mathematically rigorous way to handle the latent alignment variable.
Experimental Results
The methodology was evaluated on the AMR 2.0 and AMR 3.0 benchmarks. AMR 3.0 is particularly challenging as it includes diverse genres such as Wikipedia and LORELEI data.
Alignment Quality
Intrinsic evaluation on a gold-aligned test set showed that the neural aligner achieved a 96.5 F1 score. A key finding was the impact of pretraining; without ELMo embeddings, the F1 score dropped to approximately 80, demonstrating that contextual word representations are essential for high-quality alignment induction.
Parsing Performance
The parser achieved a Smatch score of 83.1 on AMR 3.0. This is a significant result for several reasons:
- Gold-Only Efficiency: The model reached state-of-the-art levels using only gold-standard training data, matching the performance of previous systems that required massive amounts of synthetic "silver" data.
- Decoding Speed: Unlike many top-performing parsers that require computationally expensive beam search, this model achieves its results using simple greedy decoding, making it much faster for real-world applications.
- Generalization: The neural approach proved far more robust than rule-based pipelines when moving from AMR 2.0 to the more diverse AMR 3.0 dataset.
Conclusion
By treating alignment as a probabilistic latent variable rather than a fixed preprocessing step, this work demonstrates that transition-based AMR parsers can become both more accurate and more efficient. The transition from brittle, rule-based pipelines to uncertainty-aware neural models allows for better generalization across domains and reduces the reliance on synthetic data. Future work may explore hybrid training strategies that combine the strengths of Posterior Regularization and Importance Sampling to further refine the integration between aligners and parsers.
Summary updated