Abstract
In this paper, we demonstrate the importance of coreference resolution for natural language processing on the example of the TAC Slot Filling shared task. We illustrate the strengths and weaknesses of automatic coreference resolution systems and provide experimental results to show that they improve performance in the slot filling end-to-end setting. Finally, we publish KBPchains, a resource containing automatically extracted coreference chains from the TAC source corpus in order to support other researchers working on this topic.
Introduction
In Information Extraction (IE), Slot Filling (SF) is the task of automatically populating a database with facts about specific entities—for instance, identifying the "founder" of "Apple" or the "birthplace" of "Steve Jobs." A primary challenge in this domain is the variability of human language. A single document might mention "Steve Jobs" once, but subsequently refer to him using pronouns ("he"), titles ("the Apple co-founder"), or descriptive phrases ("the 56-year-old").
Without Coreference Resolution (CR)—the process of linking these disparate mentions into a single "chain"—an extraction system relying on literal string matching will miss a significant portion of relevant data. However, CR is computationally expensive and prone to error propagation. This study quantifies whether the performance gains from CR justify its integration into end-to-end SF pipelines.
System Architecture
The authors integrated a coreference resolution module into a standard Slot Filling pipeline. The system architecture follows a modular flow designed to handle large-scale document corpora.
As shown in the pipeline, the process begins with Query Expansion to identify aliases for the target entity. This is followed by Information Retrieval from a massive corpus. The Candidate Extraction phase is where CR is applied; the system uses Stanford CoreNLP to identify mentions and link them to the target entity. Finally, Candidate Classification and Postprocessing determine the validity of the extracted fillers and clean the output for the final database.
The KBPchains Resource
To address the high computational cost of running CR on the fly, the authors pre-processed the entire TAC source corpus (over 2 million documents). The resulting resource, KBPchains, contains approximately 54 million coreference chains and 197 million mentions. By providing these pre-calculated links, the authors enable other researchers to bypass the "slowness" bottleneck of CR in their own SF experiments.
Experimental Results
The impact of CR was evaluated using the TAC 2015 Slot Filling shared task data. The results demonstrate that coreference resolution is a critical component for high-performing IE systems.
The integration of CR led to a substantial 6.54-point increase in the F1 score. Notably, the most significant improvement was seen in Precision, which jumped by over 12 points (from 19.33 to 31.67). This suggests that CR does more than just find more facts (Recall); it helps the system correctly attribute information to the right entity by resolving ambiguous pronouns and descriptions.
Discussion and Limitations
While the benefits are clear, the authors identified several persistent challenges:
- Web Data Noise: HTML artifacts in web documents often cause CR systems to incorrectly cluster unrelated entities.
- The "Anchor" Problem: Many coreference chains consist entirely of pronouns (e.g., "he... him... his") without a named entity "anchor," making them impossible to link to a specific query entity.
- Nominal Anaphors: Current systems still struggle with complex descriptions like "the author" or "the company," which are often left unlinked.
Despite these hurdles, the 6.54 F1 gain confirms that coreference resolution is an essential, albeit complex, component of the modern information extraction toolkit.
Summary updated