Event Coreference Resolution: A Survey of Two Decades of Research

Jing Lu, Vincent NgIJCAI 2018 · DOI

Introduction

Event Coreference Resolution (ECR) is the task of determining whether multiple mentions in a text refer to the same real-world event. While related to Entity Coreference—identifying that "Barack Obama" and "the 44th President" are the same person—ECR is significantly more complex. For instance, determining if "the 2008 election" and "when Obama was voted into office" refer to the same occurrence requires deep semantic understanding.

ECR serves as a critical component for high-level Information Extraction (IE) tasks, including document summarization, question answering, and knowledge base population. However, it faces two primary challenges. First, event mentions are linguistically diverse, appearing as verbs, gerunds, or entire clauses. Second, ECR typically relies on a complex pipeline of preceding tasks, such as entity extraction and argument detection. This dependency creates a "noisy game of telephone" where errors in early stages propagate through the system.

The standard pipeline architecture for event coreference resolution, illustrating the dependency on entity extraction and argument detection.

Methodological Evolution

Over the last twenty years, the field has shifted through several architectural paradigms:

  • Mention-Pair Models: These traditional baselines use binary classifiers to decide if two mentions are coreferent, followed by clustering techniques like transitive closure.
  • Mention-Ranking Models: These models improve on pair-wise decisions by ranking all preceding candidate mentions to find the most likely antecedent, capturing broader context.
  • Easy-First (Sieve) Models: These employ a sequence of rules or classifiers ordered by precision. By making "easy" links first (e.g., exact string matches), the system builds a foundation to resolve more ambiguous cases.
  • Joint Models: Representing the modern state-of-the-art, joint models attempt to solve multiple tasks—such as entity extraction and event coreference—simultaneously. This allows components to mutually inform and correct one another, mitigating the pipeline error propagation problem.

Datasets and Benchmarks

Several major corpora have shaped ECR research, each with different focuses:

  • MUC (Message Understanding Conferences): Focused on specific scenarios like terrorist attacks.
  • ACE (Automatic Content Extraction): The long-standing standard for within-document coreference.
  • OntoNotes: A large-scale corpus primarily annotating nominal (noun-based) event mentions.
  • ECB/ECB+ (EventCorefBank): The current standard for cross-document coreference, linking events across different news articles.
  • KBP (Knowledge Base Population): The most recent and complex benchmark, featuring hierarchical ontologies and multilingual data (English, Chinese, Spanish).

Key Insights

Research has consistently highlighted three critical factors in ECR performance:

  1. The "Oracle" Gap: There is a massive performance disparity between systems using "Gold" mentions (human-annotated) and "System" mentions (AI-detected). Performance often drops by over 30 points when the system must identify its own event triggers.
  2. The Role of Arguments: Resolving events is nearly impossible without considering participants (who, where, when). Temporal and spatial discrepancies are strong indicators that semantically similar words refer to different events.
  3. Semantic Embeddings: Modern models have largely moved from hand-crafted features to word embeddings, which now provide the most significant weight in coreference decisions.

Performance and Results

As of 2018, performance varies significantly by language and task setup. In English, systems on the ECB+ dataset achieve a CoNLL F1 score of approximately 68.9. However, multilingual performance remains a major challenge. For example, on the KBP 2017 dataset, Spanish systems achieved an average F1 of only 6.55, indicating a severe need for better cross-lingual transfer methods.

Performance comparison of state-of-the-art ECR systems across different languages and corpora, showing the significant drop in multilingual settings and the gap between gold and system mentions.

Furthermore, trigger detection—the prerequisite task of finding the event in the text—peaks around 69.1 F1, meaning nearly 30% of events are missed before coreference resolution even begins.

Future Directions

Despite two decades of progress, several areas remain underdeveloped:

  • Computational Scalability: While joint models are effective, they are computationally expensive and difficult to scale to large document collections.
  • Low-Resource Adaptation: Most research remains English-centric. Developing unsupervised or semi-supervised methods for other languages is a priority.
  • Partial Coreference: Most current systems only model identity (A = B). Future work must address more nuanced relationships, such as sub-events (a "shooting" within a "robbery") or membership (individual "protests" within a larger "movement").

Summary updated

BibTeX
@inproceedings{lu2018event,
  author = {Jing Lu and Vincent Ng},
  title = {{Event Coreference Resolution: A Survey of Two Decades of Research}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/773},
}
Figure