Abstract
AM dependency parsing is a method for neural semantic graph parsing that exploits the principle of compositionality. While AM dependency parsers have been shown to be fast and accurate across several graphbanks, they require explicit annotations of the compositional tree structures for training. In the past, these were obtained using complex graphbank-specific heuristics written by experts. Here we show how they can instead be trained directly on the graphs with a neural latent-variable model, drastically reducing the amount and complexity of manual heuristics. We demonstrate that our model picks up on several linguistic phenomena on its own and achieves comparable accuracy to supervised training, greatly facilitating the use of AM dependency parsing for new sembanks.
Introduction
Semantic graph parsing is the task of translating natural language into structured meaning representations, such as Abstract Meaning Representation (AMR). This process often relies on the principle of compositionality—the idea that the meaning of a complex expression is built from the meanings of its constituent parts.
The AM (Apply-Modify) dependency parser is a powerful tool for this task. However, it faces a significant "chicken and egg" problem: training the parser requires "AM dependency trees," which are structured blueprints detailing how graph fragments combine. Standard datasets, or graphbanks, only provide the final semantic graphs, not these instructional trees. Historically, researchers had to develop complex, dataset-specific heuristics to reverse-engineer these trees, a process that could take months and hindered the application of AM parsing to new languages or frameworks.
Methods
To overcome the manual bottleneck, the authors introduce a "decomposition-to-automata" pipeline that automates the discovery of hidden compositional structures. The approach treats these structures as latent variables.
- Graph Unrolling & Resolution: A graphbank-independent algorithm transforms semantic graphs (which may contain cycles or shared nodes) into trees. This is achieved by "unrolling" the graph and employing a specialized type system to track reentrancies (shared nodes).
- Tree Automata for Source Names: In AM parsing, graph fragments connect via "sources" (e.g., Subject or Object slots). Since these labels are absent in the raw data, the authors use a bottom-up tree automaton to represent all valid labeling schemes for a given tree.
- Latent Variable Training: The neural parser is trained to maximize the total probability of all valid trees that evaluate to the target graph. This is implemented using the inside-outside algorithm, allowing the model to discover the most consistent labeling schemes across the corpus.
Key Insights and Results
A significant finding is the model's emergence of syntactic consistency. Without explicit instruction, the model learns to use the same "source" for a sentence's subject, regardless of whether the verb is active or passive. This occurs because the neural parser finds it easier to predict labels that align with the surface syntax it observes in the input string.
The "learned" parser matches or slightly exceeds the performance of previous versions reliant on expert rules:
- SDP Datasets: Achieved F1 scores of 94.5 (DM), 94.8 (PAS), and 82.7 (PSD), on par with supervised state-of-the-art models.
- AMR 2.0: Achieved a 76.5 Smatch score, demonstrating its capability with complex, non-isomorphic structures.
By utilizing latent variables, the model is free to find compositional structures that are inherently easier for a neural network to learn, often leading to better generalization than hand-written rules.
Conclusion and Future Work
This work significantly reduces the expert knowledge required for AM parsing, narrowing it down to a single basic heuristic: partitioning graph edges to nodes (the "blob" partition). Future research could focus on learning this partition directly. Additionally, while the model handles AMR well, specialized treatment of coreference—which is non-compositional—could further improve scores. Although training on tree automata increases computational complexity, the resulting inference remains fast, making compositional semantic parsers more accessible for the NLP community.
Summary updated