CorefPrompt: Prompt-based Event Coreference Resolution by Measuring Event Type and Argument Compatibilities

Sheng Xu, Peifeng Li, Qiaoming Zhu

Abstract

Event coreference resolution (ECR) aims to group event mentions referring to the same real-world event into clusters. Most previous studies adopt the "encoding first, then scoring" framework, making the coreference judgment rely on event encoding. Furthermore, current methods struggle to leverage human-summarized ECR rules, e.g., coreferential events should have the same event type, to guide the model. To address these two issues, we propose a prompt-based approach, CorefPrompt, to transform ECR into a cloze-style MLM (masked language model) task. This allows for simultaneous event modeling and coreference discrimination within a single template, with a fully shared context. In addition, we introduce two auxiliary prompt tasks, event-type compatibility and argument compatibility, to explicitly demonstrate the reasoning process of ECR, which helps the model make final predictions. Experimental results show that our method CorefPrompt performs well in a state-of-the-art (SOTA) benchmark.

Introduction

Event Coreference Resolution (ECR) is a fundamental task in natural language processing that involves identifying whether different mentions in a text refer to the same real-world occurrence. For instance, in a report describing a disaster, terms like "the explosion," "the blast," and "the incident" must be linked to the same event cluster to build an accurate representation of the narrative.

Despite its importance, ECR faces two significant challenges. First, the traditional "encode first, then score" paradigm creates an information block. In this framework, an encoder (such as BERT) generates representations for event mentions independently before a separate classifier determines their relationship. This prevents the model from capturing interactive, context-specific nuances during the encoding phase. Second, ECR is governed by logical rules—such as the requirement that coreferential events share compatible types and arguments (who, where, when)—which are difficult to explicitly integrate into standard neural architectures.

The CorefPrompt Framework

To overcome these hurdles, we introduce CorefPrompt, a method that reimagines ECR as a cloze-style Masked Language Modeling (MLM) task. Instead of treating coreference as a binary classification problem, CorefPrompt uses structured templates to guide the model through a logical reasoning process.

Template-Based Reasoning

The framework utilizes a multi-stage prompt structure to process event pairs:

  1. Prefix Template: Directs the model's attention to specific event triggers within the source text.
  2. Anchor Templates: These are inserted around event mentions to explicitly identify participants and locations. They also predict event types using "virtual label words"—mathematically derived embeddings that represent complex event categories (e.g., Conflict.Attack).
  3. Inference Template: This acts as a "Chain of Thought" (CoT) engine. It prompts the model to fill masks corresponding to three sequential questions:
  4. Are the event types compatible?
  5. Are the event arguments compatible?
  6. Based on the above, do these mentions refer to the same event?

Mask Token Embedding Updating

To further enhance the interaction between event mentions, we implement a Mask Token Embedding Updating technique. This involves calculating tensor similarities between event representations and injecting these interactive features directly into the [MASK] tokens. This ensures that the model's final coreference judgment is informed by a deep, shared understanding of both mentions.

Experimental Setup

We evaluated CorefPrompt using the standard Knowledge Base Population (KBP) benchmarks. The training set comprised KBP 2015 and KBP 2016 (817 documents, ~23k mentions), while KBP 2017 (167 documents, ~4.4k mentions) served as the test set.

Given the massive number of potential event pairs, we employed a specialized undersampling strategy known as CorefNM. This strategy prioritizes "hard" negative examples—pairs that are similar but not coreferential—to challenge the model and improve its discriminative power during training.

Results and Analysis

CorefPrompt achieves state-of-the-art performance while maintaining high computational efficiency. Key findings include:

  • State-of-the-Art Performance: CorefPrompt reached an AVG-F1 score of 51.3 on the KBP 2017 dataset, surpassing previous benchmarks. Notably, it achieved this without requiring full-text encoding, which is computationally expensive.
  • Prompting vs. Fine-tuning: The prompt-based MLM approach significantly outperformed standard pairwise fine-tuning (which scored 48.6). This suggests that the MLM structure is more effective at leveraging the latent knowledge within pre-trained language models for relational tasks.
  • Ablation Study: Removing the auxiliary tasks (type and argument compatibility) led to a drop of over 2 points in the F1 score. This confirms that explicitly modeling the reasoning steps is crucial for accurate coreference resolution.
  • Efficiency: Unlike many SOTA models that require high-end industrial GPUs (e.g., 80GB A100s), CorefPrompt’s segment-level encoding allows it to run effectively on consumer-grade hardware, such as a 24GB RTX 3090.

Conclusion

CorefPrompt demonstrates that transforming Event Coreference Resolution into a prompt-based reasoning task effectively addresses the limitations of traditional encoding-scoring frameworks. By explicitly measuring event type and argument compatibility within a shared context, the model mimics human-like logical deduction. Future work will focus on moving beyond the current pipeline dependency—where triggers must be identified before coreference can be resolved—toward a unified system that performs both tasks simultaneously.

Summary updated

Abstract

Event coreference resolution (ECR) aims to group event mentions referring to the same real-world event into clusters. Most previous studies adopt the "encoding first, then scoring" framework, making the coreference judgment rely on event encoding. Furthermore, current methods struggle to leverage human-summarized ECR rules, e.g., coreferential events should have the same event type, to guide the model. To address these two issues, we propose a prompt-based approach, CorefPrompt, to transform ECR into a cloze-style MLM (masked language model) task. This allows for simultaneous event modeling and coreference discrimination within a single template, with a fully shared context. In addition, we introduce two auxiliary prompt tasks, event-type compatibility and argument compatibility, to explicitly demonstrate the reasoning process of ECR, which helps the model make final predictions. Experimental results show that our method CorefPrompt performs well in a state-of-the-art (SOTA) benchmark.
BibTeX
@inproceedings{xu2023corefprompt,
  author = {Sheng Xu and Peifeng Li and Qiaoming Zhu},
  title = {{CorefPrompt: Prompt-based Event Coreference Resolution by Measuring Event Type and Argument Compatibilities}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=mDrx5kowNd},
}
Figure