Paraphrasing vs Coreferring: Two Sides of the Same Coin

Yehudit Meged, Avi Caciularu, Vered Shwartz, Ido Dagan

Abstract

We study the potential synergy between two different NLP tasks, both confronting predicate lexical variability: identifying predicate paraphrases, and event coreference resolution. First, we used annotations from an event coreference dataset as distant supervision to re-score heuristically-extracted predicate paraphrases. The new scoring gained more than 18 points in average precision upon their ranking by the original scoring method. Then, we used the same re-ranking features as additional inputs to a state-of-the-art event coreference resolution model, which yielded modest but consistent improvements to the model's performance. The results suggest a promising direction to leverage data and models for each of the tasks to the benefit of the other.

Introduction

The challenge of predicate lexical variability—where different expressions describe the same action (e.g., "checked into" vs. "entered")—is a central hurdle in NLP. Historically, this has been addressed through two independent tasks: Paraphrase Identification, which finds generic lexical matches, and Cross-Document (CD) Event Coreference Resolution, which identifies specific real-world event matches in context. This research investigates the synergy between these tasks, positing that they are "two sides of the same coin" and can mutually benefit from shared data and modeling techniques.

Methods

The authors propose a bidirectional framework for information sharing between these tasks.

1. Improving Paraphrasing with Coreference (Chirps*)

Starting with Chirps, a noisy unsupervised resource of predicate paraphrases from Twitter, the authors developed Chirps*. This is a supervised Random Forest classifier that re-ranks candidate pairs using 17 features, including Named Entity Coverage (overlap in linked news articles), CD coreference model predictions, and graph-based features like connected components and cliques. These graph structures help identify predicates that are used interchangeably across many distinct real-world events.

2. Improving Coreference with Paraphrasing

To enhance coreference resolution, the 17-dimensional feature vector from Chirps* is integrated into a state-of-the-art CD coreference model (Barhom et al., 2019). As illustrated in Figure 1, these features are passed through a Multi-Layer Perceptron (MLP) and concatenated with the model's standard mention-pair representations to refine pairwise scoring.

Architecture of the enhanced coreference model integrating Chirps* features.

Datasets

The study leverages the ECB+ benchmark, using its coreference annotations as distant supervision to label training data for the paraphrase scorer. Additionally, the authors introduced ch-ECB+ and ch-Random, subsets of Chirps validated via crowdsourcing to evaluate the Chirps* ranker.

Results

Paraphrase Ranking Performance

The supervised Chirps scorer achieved a massive improvement in ranking quality. On the ch-ECB+ test set, Average Precision (AP) reached 80.0*, a substantial jump from the 62.5 achieved by the original Chirps heuristic and 50.7 for a GloVe baseline. These results are summarized in the table below.

Performance comparison of paraphrase ranking models on the ch-ECB+ and ch-Random datasets.

Coreference Resolution Performance

Incorporating Chirps features into the coreference model yielded modest but consistent gains. The CoNLL $F_1$ score improved from 79.5 to 80.0, and the MUC $F_1$ score rose from 80.9 to 81.61*. The authors noted that gains were likely constrained by a coverage mismatch, as Chirps covers only about 30% of mentions in the ECB+ dataset.

Conclusion

This work demonstrates that coreference annotations serve as an excellent filter for cleaning noisy paraphrase datasets, while paraphrase features provide valuable signals for coreference resolution. Future research will focus on developing fully joint models to further exploit the synergistic relationship between these two tasks.

Summary updated

Abstract

We study the potential synergy between two different NLP tasks, both confronting predicate lexical variability: identifying predicate paraphrases, and event coreference resolution. First, we used annotations from an event coreference dataset as distant supervision to re-score heuristically-extracted predicate paraphrases. The new scoring gained more than 18 points in average precision upon their ranking by the original scoring method. Then, we used the same re-ranking features as additional inputs to a state-of-the-art event coreference resolution model, which yielded modest but consistent improvements to the model's performance. The results suggest a promising direction to leverage data and models for each of the tasks to the benefit of the other.
BibTeX
@inproceedings{meged2020paraphrasing,
  author = {Yehudit Meged and Avi Caciularu and Vered Shwartz and Ido Dagan},
  title = {{Paraphrasing vs Coreferring: Two Sides of the Same Coin}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=mSLj4144YX1},
}
Figure