Cross-document Coreference Resolution over Predicted Mentions

Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, Ido Dagan

Abstract

Coreference resolution has been mostly investigated within a single document scope, showing impressive progress in recent years based on end-to-end models. However, the more challenging task of cross-document (CD) coreference resolution remained relatively under-explored, with the few recent models applied only to gold mentions. Here, we introduce the first end-to-end model for CD coreference resolution from raw text, which extends the prominent model for within-document coreference to the CD setting. Our model achieves competitive results for event and entity coreference resolution on gold mentions. More importantly, we set first baseline results, on the standard ECB+ dataset, for CD coreference resolution over predicted mentions. Further, our model is simpler and more efficient than recent CD coreference resolution systems, while not using any external resources.

Introduction

Coreference resolution is the task of identifying when different text spans refer to the same underlying entity or event—for instance, linking "the president" to "Barack Obama" across different documents. While within-document (WD) coreference has advanced significantly through end-to-end neural models, cross-document (CD) coreference has lagged behind, often relying on several unrealistic assumptions in existing research.

Most current CD coreference models evaluate exclusively on gold mentions (pre-identified perfect spans), rely heavily on external dependencies like Semantic Role Labeling (SRL) or paraphrase dictionaries, and suffer from computational inefficiency due to recomputing heavy transformer encodings for every possible mention pair. This paper bridges these gaps by proposing a system that handles the entire pipeline—from mention detection to cross-document clustering—efficiently and without external resources.

Methodology

Our approach builds upon the prominent e2e-coref model (Lee et al., 2017), adapting it for the non-linear nature of cross-document relationships. The pipeline consists of four main stages:

The end-to-end model pipeline for cross-document coreference resolution.
  1. Span Extraction & Pruning: Using a frozen RoBERTa-Large encoder, the model extracts all possible text spans and uses a learned "mention scorer" to prune unlikely candidates.
  2. Pre-training: To address the small size of CD datasets, the mention scorer is pre-trained on gold spans to help it generate high-quality candidate spans early in training.
  3. Global Pairwise Scoring: Unlike WD models that look sequentially backward, this model uses a Multi-Layer Perceptron (MLP) to compare a candidate mention to all other spans across the document cluster.
  4. Agglomerative Clustering: Average-linkage agglomerative clustering is applied over pairwise scores to group mentions into final coreference chains.

Experimental Results

We evaluate our model on the ECB+ dataset, the primary benchmark for CD coreference. ECB+ contains 43 topics describing lexically similar but distinct sub-events to test a model's ability to handle ambiguity.

Performance comparison on the ECB+ dataset for event and entity coreference across various models and settings.

Gold vs. Predicted Mentions

On gold mentions, our model achieves competitive CoNLL F1 scores of 81.0 for events and 73.1 for entities. However, moving to predicted mentions (raw text) reveals a steep performance drop, with F1 scores falling to 54.4 for events and 35.7 for entities. This highlights that mention detection is a severe bottleneck in real-world CD coreference, a challenge often obscured by previous literature that focused only on gold mentions.

Efficiency and Unified Modeling

Our model is highly efficient; inference over the test set takes just 11 minutes. In head-to-head hardware comparisons, the authors estimate their model runs in roughly 2 minutes, compared to 37 minutes for previous state-of-the-art systems. Additionally, a unified "ALL" model that resolves events and entities jointly performs nearly as well as specialized models (43.4 F1 on predicted mentions), simplifying the overall pipeline.

Discussion and Future Work

A key insight from our experiments is the "singleton penalty." Approximately 10% of the performance drop on raw text is due to CD evaluation metrics including "singletons" (mentions appearing only once), which standard WD systems typically ignore.

While our model is significantly faster than prior work, it still requires $O(n^2)$ pairwise comparisons. Future work should explore scalable hierarchical clustering or parallel processing to apply these methods to massive, corpus-level datasets. Improving performance on raw text may also require incorporating semantic information, such as SRL, to better identify event participants or jointly optimizing mention detection and coreference linking.

Summary updated

Abstract

Coreference resolution has been mostly investigated within a single document scope, showing impressive progress in recent years based on end-to-end models. However, the more challenging task of cross-document (CD) coreference resolution remained relatively under-explored, with the few recent models applied only to gold mentions. Here, we introduce the first end-to-end model for CD coreference resolution from raw text, which extends the prominent model for within-document coreference to the CD setting. Our model achieves competitive results for event and entity coreference resolution on gold mentions. More importantly, we set first baseline results, on the standard ECB+ dataset, for CD coreference resolution over predicted mentions. Further, our model is simpler and more efficient than recent CD coreference resolution systems, while not using any external resources.
BibTeX
@inproceedings{cattan2021cross,
  author = {Arie Cattan and Alon Eirew and Gabriel Stanovsky and Mandar Joshi and Ido Dagan},
  title = {{Cross-document Coreference Resolution over Predicted Mentions}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.453},
}
Figure