Adapting Coreference Resolution for Narrative Processing

Quynh Ngoc Thi Do, Steven Bethard, Marie-Francine Moens

Introduction

Coreference resolution—the task of determining when different words or phrases refer to the same entity—is a fundamental component of natural language understanding. However, state-of-the-art systems often struggle when moved from their training domain (typically newswire) to narrative domains like folktales or historical texts. Performance often plummets because the linguistic patterns in a financial news report differ significantly from those in a Grimm’s Fairy Tale.

To address this without the prohibitive cost of manual re-annotation, researchers have developed a method for unsupervised domain adaptation. By using Integer Linear Programming (ILP) to inject narrative-specific linguistic rules as "soft constraints," they have successfully adapted "news-smart" models to be "story-smart" without retraining the underlying architecture.

Methodology

The approach builds upon the Berkeley coreference system, a log-linear model that usually makes local decisions about antecedent links. The researchers shifted the final decision-making into an Integer Linear Programming (ILP) framework. This allows the system to treat the original model's scores as a baseline while layering on narrative-specific constraints. If a proposed link violates a narrative rule, the ILP objective function applies a penalty, effectively steering the model toward more linguistically sound decisions for the domain.

Three categories of constraints were introduced:

  1. Centering Theory: Rules modeling how the "topic" of a story shifts between sentences.
  2. Direct Speech: Logic to handle dialogue, linking first-person pronouns to speakers and second-person pronouns to listeners.
  3. Character Naming: Rules linking indefinite introductions (e.g., "a wolf") to subsequent definite mentions (e.g., "the wolf") and enforcing exact string matching for names.

Experimental Results

The system was evaluated on two narrative-heavy datasets: UMIREC (Tales), consisting of 30 English folktales, and N2 (Hadith), containing 64 historical narratives. The ILP-based inference (ILPI) consistently outperformed the standard Berkeley (BER) inference across both datasets.

Performance comparison of ILPI vs BER on UMIREC and N2 datasets.

As shown in the results table, the average CoNLL metric on UMIREC rose from 61.34 to 66.76 when using gold mentions. Similar gains were observed on the N2 dataset. Even when the system had to predict mentions (a significantly harder task), the ILP method provided a steady improvement of approximately 1-2 points.

Analysis of Constraints

A significant insight from this work is that the most powerful constraints are often the simplest. While complex discourse theories like Centering Theory are helpful, the largest performance boosts came from enforcing consistency in character naming and definite noun phrases.

Impact of individual constraints on coreference performance.

In narratives, characters are frequently introduced once and then referred to by the same name or title repeatedly. Capturing this global consistency through "Definite & Exact" constraints proved more impactful than more complex linguistic modeling. Because these are "soft" constraints, the ILP can still choose to violate a rule if the underlying model is extremely confident, ensuring the system remains robust to noise.

Conclusion and Future Work

This research demonstrates that inference-time adaptation is a powerful alternative to model retraining. By changing how a model chooses its final answer rather than changing its internal weights, domain knowledge can be injected post-hoc. While the current system requires a small amount of held-out data to tune penalty parameters, it represents a major step toward flexible, domain-aware coreference resolution. Future work may address the computational overhead of ILP solvers and explore the automatic discovery of relevant constraints for new domains.

Summary updated

Abstract

Domain adaptation is a challenge for supervised NLP systems because of expensive and time-consuming manual annotated resources. We present a novel method to adapt a supervised coreference resolution system trained on newswire to short narrative stories without retraining the system. The idea is to perform inference via an Integer Linear Programming (ILP) formulation with the features of narratives adopted as soft constraints. When testing on the UMIREC 1 and N2 2 corpora with the-stateof-the-art Berkeley coreference resolution system trained on OntoNotes 3 , our inference substantially outperforms the original inference on the CoNLL 2011 metric.
BibTeX
@inproceedings{do2015adapting,
  author = {Quynh Ngoc Thi Do and Steven Bethard and Marie-Francine Moens},
  title = {{Adapting Coreference Resolution for Narrative Processing}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/d15-1271},
}
Figure