Predicting Coreference in Abstract Meaning Representations

Tatiana Anikina, Alexander Koller, Michael RothUniversität 2020 · DOI

TL;DR

This paper tackles the challenge of coreference resolution within Abstract Meaning Representations (AMRs), a structured way to represent sentence meaning. By evaluating existing coreference tools on a new multi-sentence AMR corpus, they establish baselines and show that incorporating coreference information significantly improves the performance of state-of-the-art AMR parsers.

What Problem Does This Paper Address?

Abstract Meaning Representations (AMRs) are powerful symbolic representations of sentence meaning, often used in semantic parsing. However, standard AMR representations primarily focus on a single sentence and struggle to capture relationships between entities or concepts that span across multiple sentences. Coreference resolution, the task of identifying mentions that refer to the same real-world entity, is crucial for understanding discourse. This paper addresses the gap of applying and evaluating coreference resolution techniques specifically within the context of AMR graphs, particularly for multi-sentence contexts, and demonstrating its benefit for AMR parsing.

Methods

The authors evaluate existing coreference resolution tools, specifically AllenNLP with GloVe and SpanBERT embeddings, and two versions of CoreNLP (deterministic and neural). They adapt these tools to work with the MS-AMR corpus by considering two evaluation modes:

  • Token-level: Projecting AMR coreference annotations to text tokens and comparing them with the tools' predictions.
  • Node-level: Projecting the tools' token-level predictions onto the AMR graph nodes and comparing them with the gold AMR coreference annotations.

They also integrate a state-of-the-art AMR parser (Lindemann et al., 2019) with predicted coreference information. This involves:

  1. Creating document-level gold AMR annotations by merging sentence-level AMRs and coreference chains.
  2. Running the AMR parser sentence-by-sentence and then merging the predicted graphs.
  3. Mapping token-level coreference predictions from AllenNLP to the parser's predicted nodes.
  4. Collapsing coreferent nodes in the AMR graph by redirecting edges to a single representative node.

Datasets

The primary dataset used is the Multi-Sentence AMR (MS-AMR) corpus. This corpus extends the AMR-2017 graphbank by adding a layer of annotation for coreference and implicit arguments beyond the sentence level. Unlike traditional coreference datasets that focus on text spans, MS-AMR annotates coreference between nodes within AMR graphs.

Key Insights and Ideas

  • Node-level coreference is key for AMR: The MS-AMR corpus highlights that coreference in AMR isn't always a direct one-to-one mapping with text tokens. Some AMR nodes can be part of coreference chains without explicit token alignment, and multiple tokens might map to a single node. This necessitates evaluating coreference resolution at the graph node level, not just the token level.
  • SpanBERT shines in AMR coreference: Among the evaluated tools, AllenNLP with SpanBERT embeddings consistently achieved the best performance across various metrics, both at the token and node levels. This suggests that contextual embeddings like SpanBERT are particularly effective for capturing the nuances of coreference in AMR.
  • Coreference significantly boosts AMR parsing: Integrating predicted coreference information into a state-of-the-art AMR parser led to a substantial improvement of approximately three points in Smatch f-score. This demonstrates that understanding cross-sentence entity references is vital for accurate semantic parsing.
  • Heterogeneous coreference chains are challenging: The MS-AMR corpus contains "heterogeneous" coreference chains where mentions of the same entity are expressed using different parts of speech (e.g., "it," "thing," "harm-01," "cut-01"). These are difficult for current models to resolve, as they require understanding semantic relationships beyond simple lexical matching.

Key Results

  • Token-level evaluation: AllenNLP with SpanBERT achieved an F-score of 0.58 for MUC, closely followed by neural CoreNLP (0.59). However, SpanBERT AllenNLP performed best overall across metrics like B³ and CEAF.
  • Node-level evaluation: AllenNLP with SpanBERT achieved the best results across all metrics (MUC, B³, CEAF) at the node level, indicating its superior performance in handling the graph-based nature of MS-AMR coreference.
  • AMR parser improvement: Augmenting the Lindemann et al. (2019) AMR parser with predicted coreference information resulted in a three-point improvement in Smatch f-score on the MS-AMR test data.
  • The MS-AMR corpus is noted as being more challenging than standard AMR datasets, with a lower overall Smatch score for parsers.

Limitations and Future Work

  • Textual information reliance: The coreference models used in this study relied primarily on textual information. The authors suggest future work could incorporate features directly from the AMR graphs themselves, which abstract away from surface-level details.
  • Bridging coreference: The paper focuses on standard coreference. Future work could explore predicting and incorporating bridging coreference relations (e.g., "the car" and "its engine") into AMR parsing.
  • Challenges with heterogeneous chains: The difficulty in resolving heterogeneous coreference chains (different parts of speech referring to the same entity) remains an open challenge.
  • Pronoun resolution in conversational contexts: The paper notes issues with resolving personal pronouns in conversational texts where speakers might switch, leading to ambiguity for models lacking speaker meta-information.
  • Node-token alignment imperfections: Incomplete or incorrect node-token alignments in the MS-AMR corpus can hinder the performance of token-based coreference resolvers.

Summary updated

Abstract

This work addresses coreference resolution in Abstract Meaning Representation (AMR) graphs, a popular formalism for semantic parsing. We evaluate several current coreference resolution techniques on a recently published AMR coreference corpus, establishing baselines for future work. We also demonstrate that coreference resolution can improve the accuracy of a state-of-the-art semantic parser on this corpus.
BibTeX
@inproceedings{anikina2020predicting,
  author = {Tatiana Anikina and Alexander Koller and Michael Roth},
  title = {{Predicting Coreference in Abstract Meaning Representations}},
  booktitle = {Universität},
  year = {2020},
  doi = {10.22028/d291-36109},
}
Figure