An Integrated, Conditional Model of Information Extraction and Coreference with Applications to Citation Matching

Ben Wellner, Andrew McCallum, Fuchun Peng, Michael Hay

Introduction

In traditional natural language processing pipelines, Information Extraction (IE) and Coreference Resolution (CR) are typically treated as independent, sequential tasks. For example, a system might first identify fields like "Author" or "Title" in a citation string and then use those extracted fields to determine if two citations refer to the same publication. This serial approach suffers from error propagation—where extraction mistakes inevitably lead to coreference failures—and fails to leverage the synergy between the tasks.

This paper proposes an integrated framework using conditionally-trained undirected graphical models to perform IE and CR simultaneously. By allowing uncertainty and evidence to flow in both directions, the model can use coreference information to correct extraction errors and vice versa.

Integrated Model Architecture

The authors utilize Conditional Random Fields (CRFs) to represent the joint task. Unlike generative models, CRFs allow for the inclusion of arbitrary, overlapping features of the input text without requiring explicit modeling of their dependencies.

The integrated graphical model structure showing the relationship between observed citations (x), segmentation (s), citation fields (c), entity attributes (a), and pairwise coreference (y).

The integrated model consists of three primary components: segmentation for IE, pairwise coreference represented as a graph partitioning problem, and entity attribute variables that represent "canonical" values derived from multiple mentions. To handle the complexity of joint inference, the authors employ a structured variant of Iterated Conditional Modes (ICM). A key innovation is the use of N-best integration: instead of passing only the single best segmentation to the coreference step, the model passes the top $N$ hypotheses. This allows the coreference component to resolve extraction ambiguities by choosing the segmentation that best fits the global context of the dataset.

Experimental Evaluation

The model was evaluated on the CiteSeer dataset, which contains approximately 1,500 citations across four topical subsets: Reinforce, Face, Reason, and Constraint. The goal was to segment citations into fields and cluster those referring to the same publication.

Key Results

The integrated approach demonstrated significant improvements over baseline pipeline methods:

  • Coreference Accuracy: Using N-best lists (specifically $N=9$) resulted in a 25-35% reduction in coreference error compared to models without segmentation information.
  • Extraction Accuracy: Leveraging coreference information to refine field labels led to a 6-14% reduction in extraction error.
Experimental results showing extraction error reduction when using coreference information across four datasets.

Statistical significance was confirmed using McNemar’s test ($p < 0.05$), indicating that the gains from joint inference are robust across different citation domains.

Conclusion and Future Work

The research demonstrates that "closing the loop" between extraction and coreference provides substantial benefits. By maintaining uncertainty through N-best lists and using relational modeling, the system effectively uses the consensus of multiple noisy records to improve overall accuracy. Future work includes scaling these methods to larger datasets using more efficient sampling or variational inference and exploring more sophisticated conditional features.

Summary updated

Abstract

Although information extraction and coreference resolution appear together in many applications, most current systems perform them as ndependent steps. This paper describes an approach to integrated inference for extraction and coreference based on conditionally-trained undirected graphical models. We discuss the advantages of conditional probability training, and of a coreference model structure based on graph partitioning. On a data set of research paper citations, we show significant reduction in error by using extraction uncertainty to improve coreference citation matching accuracy, and using coreference to improve the accuracy of the extracted fields.
BibTeX
@inproceedings{wellner2004integrated,
  author = {Ben Wellner and Andrew McCallum and Fuchun Peng and Michael Hay},
  title = {{An Integrated, Conditional Model of Information Extraction and Coreference with Applications to Citation Matching}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2004},
  url = {https://openreview.net/forum?id=HTf5sNgYIZ9},
}
Figure