Coreference Resolution: Are the Eliminated Spans Totally Worthless?

Xin Tan, Longyin Zhang, Guodong ZhouLecture Notes in Computer Science 2021 · DOI · OpenAlex

Abstract

Various neural-based methods have been proposed so far for joint mention detection and coreference resolution. However, existing works on coreference resolution are mainly dependent on filtered mention representation, while other spans are largely neglected. In this paper, we aim at increasing the utilization rate of data and investigating whether those eliminated spans are totally useless, or to what extent they can improve the performance of coreference resolution. To achieve this, we propose a mention representation refining strategy where spans highly related to mentions are well leveraged using a pointer network for representation enhancing. Notably, we utilize an additional loss term in this work to encourage the diversity between entity clusters. Experimental results on the document-level CoNLL-2012 Shared Task English dataset show that eliminated spans are indeed much effective and our approach can achieve competitive results when compared with previous state-of-the-art in coreference resolution.

Introduction

Coreference resolution is the task of determining when different words or phrases in a text refer to the same real-world entity—for instance, identifying that "Elon Musk," "he," and "the CEO" all refer to the same individual. Modern neural models typically approach this through a two-stage pipeline: first, Mention Detection, where the model filters all possible text spans to identify potential mentions; and second, Clustering, where these mentions are grouped into entities.

A significant limitation of this approach is that spans discarded during the first stage are completely ignored in the second. This creates a bottleneck: if a span providing crucial context is filtered out early, that information is lost. This research investigates whether these "eliminated" spans can be leveraged to refine mention representations and improve overall resolution accuracy.

Methodology

Building upon the end-to-end architecture by Lee et al. (2017), the proposed model introduces two primary innovations to maximize data utilization:

  1. Enhanced Mention Representation via Pointer Networks: Instead of relying solely on selected mentions, the model employs a Pointer Network (PTR-NET) to revisit all candidate spans. The network learns to "point" to spans—including those previously eliminated—that are semantically related to a specific mention. This context-aware information is then merged with the mention's original representation through a gated mechanism.
  2. Diversity Loss Term: To improve cluster separation, an additional loss term is introduced. This encourages diversity between different entity clusters by learning from both positive and negative samples, effectively increasing the "distance" between distinct entities and reducing erroneous merges.
Architecture of the proposed model utilizing a Pointer Network (PTR-NET) to incorporate context from eliminated spans.

Experimental Results

The model was evaluated on the CoNLL-2012 Shared Task English dataset, the benchmark for document-level coreference resolution. The results demonstrate that re-incorporating eliminated spans leads to a measurable performance boost.

As shown in the table below, the proposed approach achieved an Average F1 score of 73.4, outperforming the baseline of 72.1. The improvements are consistent across all standard metrics: MUC, B³, and CEAFφ4. Furthermore, ablation studies (the "-additional_loss" row) confirm that the diversity loss term specifically aids the B³ indicator, helping the model distinguish between different entities more effectively.

Detailed performance comparison of the proposed model against various baselines on the CoNLL-2012 dataset.

Discussion and Insights

  • The Value of Context: Even if a span is not an entity itself (e.g., a descriptive verb phrase), it often serves as the semantic "glue" linking two mentions. By salvaging these spans, the model becomes more robust to early-stage errors in the mention detection pipeline.
  • Embedding Trade-offs: Interestingly, when using static features, ELMo embeddings were found to outperform BERT in this specific architecture. This suggests that ELMo’s shallower features may be more readily accessible for this resolution task without the need for extensive fine-tuning.
  • Future Work: While effective, using a Pointer Network to scan all spans increases computational complexity. Future research could focus on more efficient selection mechanisms for informative spans or exploring the impact of full model fine-tuning.

Conclusion

This work demonstrates that spans discarded during mention detection are far from worthless. By leveraging a Pointer Network to refine mention representations with these eliminated spans, coreference resolution models can achieve significantly higher accuracy and better entity cluster diversity.

Summary updated

BibTeX
@inproceedings{tan2021coreference,
  author = {Xin Tan and Longyin Zhang and Guodong Zhou},
  title = {{Coreference Resolution: Are the Eliminated Spans Totally Worthless?}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2021},
  doi = {10.1007/978-3-030-88480-2\_1},
}

Loading related content...

Figure