Abstract
Various neural-based methods have been proposed so far for joint mention detection and coreference resolution. However, existing works on coreference resolution are mainly dependent on filtered mention representation, while other spans are largely neglected. In this paper, we aim at increasing the utilization rate of data and investigating whether those eliminated spans are totally useless, or to what extent they can improve the performance of coreference resolution. To achieve this, we propose a mention representation refining strategy where spans highly related to mentions are well leveraged using a pointer network for representation enhancing. Notably, we utilize an additional loss term in this work to encourage the diversity between entity clusters. Experimental results on the document-level CoNLL-2012 Shared Task English dataset show that eliminated spans are indeed much effective and our approach can achieve competitive results when compared with previous state-of-the-art in coreference resolution.
Introduction
Coreference resolution is the task of determining when different words or phrases in a text refer to the same real-world entity—for instance, identifying that "Elon Musk," "he," and "the CEO" all refer to the same individual. Modern neural models typically approach this through a two-stage pipeline: first, Mention Detection, where the model filters all possible text spans to identify potential mentions; and second, Clustering, where these mentions are grouped into entities.
A significant limitation of this approach is that spans discarded during the first stage are completely ignored in the second. This creates a bottleneck: if a span providing crucial context is filtered out early, that information is lost. This research investigates whether these "eliminated" spans can be leveraged to refine mention representations and improve overall resolution accuracy.
Methodology
Building upon the end-to-end architecture by Lee et al. (2017), the proposed model introduces two primary innovations to maximize data utilization:
- Enhanced Mention Representation via Pointer Networks: Instead of relying solely on selected mentions, the model employs a Pointer Network (PTR-NET) to revisit all candidate spans. The network learns to "point" to spans—including those previously eliminated—that are semantically related to a specific mention. This context-aware information is then merged with the mention's original representation through a gated mechanism.
- Diversity Loss Term: To improve cluster separation, an additional loss term is introduced. This encourages diversity between different entity clusters by learning from both positive and negative samples, effectively increasing the "distance" between distinct entities and reducing erroneous merges.
Experimental Results
The model was evaluated on the CoNLL-2012 Shared Task English dataset, the benchmark for document-level coreference resolution. The results demonstrate that re-incorporating eliminated spans leads to a measurable performance boost.
As shown in the table below, the proposed approach achieved an Average F1 score of 73.4, outperforming the baseline of 72.1. The improvements are consistent across all standard metrics: MUC, B³, and CEAFφ4. Furthermore, ablation studies (the "-additional_loss" row) confirm that the diversity loss term specifically aids the B³ indicator, helping the model distinguish between different entities more effectively.
Discussion and Insights
- The Value of Context: Even if a span is not an entity itself (e.g., a descriptive verb phrase), it often serves as the semantic "glue" linking two mentions. By salvaging these spans, the model becomes more robust to early-stage errors in the mention detection pipeline.
- Embedding Trade-offs: Interestingly, when using static features, ELMo embeddings were found to outperform BERT in this specific architecture. This suggests that ELMo’s shallower features may be more readily accessible for this resolution task without the need for extensive fine-tuning.
- Future Work: While effective, using a Pointer Network to scan all spans increases computational complexity. Future research could focus on more efficient selection mechanisms for informative spans or exploring the impact of full model fine-tuning.
Conclusion
This work demonstrates that spans discarded during mention detection are far from worthless. By leveraging a Pointer Network to refine mention representations with these eliminated spans, coreference resolution models can achieve significantly higher accuracy and better entity cluster diversity.
Summary updated