Corpus-Based Learning for Noun Phrase Coreference Resolution

Wee Meng Soon, Hwee Tou Ng, Chung Yong LimEMNLP 1999 · OpenReview

Abstract

Wee Meng Soon, Hwee Tou Ng, Chung Yong Lim. 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora.

1999.

Introduction

Coreference resolution—the task of determining when different expressions in a text refer to the same real-world entity—is a fundamental challenge in natural language processing (NLP). In the late 1990s, this task was primarily addressed through hand-crafted, rule-based systems. This paper marks a significant shift toward machine learning, demonstrating that a supervised learning approach using a C4.5 decision tree can achieve performance comparable to state-of-the-art manual systems. Unlike earlier work focused narrowly on anaphora resolution (linking pronouns), this study tackles general noun phrase coreference across names, descriptions, and pronouns.

Methodology

The authors proposed a two-stage supervised learning framework. First, a standard NLP pipeline handles preprocessing tasks such as sentence segmentation, tokenization, part-of-speech tagging, and noun phrase identification using Hidden Markov Models (HMMs). Second, a C4.5 decision tree classifier is trained on feature vectors derived from pairs of noun phrases (markables).

Training examples were constructed by pairing an anaphor with its closest preceding antecedent (positive examples) and with all non-referring markables appearing between them (negative examples). The system utilized ten domain-independent linguistic features, including:

  • Lexical: String matching and alias detection (e.g., "IBM" vs. "International Business Machines").
  • Grammatical: Gender and number agreement, and identification of definite/demonstrative noun phrases.
  • Semantic: Semantic class matching (e.g., "Person," "Organization") based on WordNet.
  • Positional: Sentence distance between markables.
The C4.5 decision tree learned from the training corpus. The model effectively pruned the initial ten features down to five key indicators: string matching, pronoun identification, gender agreement, alias detection, and distance.

Experimental Setup and Results

The system was evaluated using the MUC-6 (Message Understanding Conference) coreference corpus, consisting of 30 training documents and 30 test documents. This allowed for a direct comparison with established rule-based systems.

The learning-based system achieved a Recall of 52% and a Precision of 68%, resulting in an F-measure of 58.9%. While these scores are lower than modern benchmarks, they were highly competitive at the time, matching the performance of top-tier manual systems and significantly outperforming previous learning-based attempts that relied on domain-specific features.

The learning curve illustrates the relationship between the number of training documents and the resulting F-measure. The steady upward trend suggests that performance had not yet plateaued at 30 documents.

Discussion and Key Insights

A primary finding was the effectiveness of a small set of shallow features. Although ten features were provided, the decision tree primarily relied on five: String Matching, Pronoun Anaphor, Gender Agreement, Alias, and Distance. This simplicity suggests that coreference resolution can be robustly handled by general linguistic indicators rather than complex, domain-specific rules.

However, the authors identified several bottlenecks:

  1. Pipeline Dependency: Approximately 15% of coreference chains were lost due to errors in the initial noun phrase extraction stage, highlighting the impact of error propagation.
  2. Semantic Limitations: Accurately determining semantic class and gender agreement remains difficult for complex entities (e.g., "the government").
  3. Data Scarcity: The learning curve indicates that the model's performance was still improving as more data was added, suggesting that a larger corpus would yield better results.

Conclusion

This work proved that corpus-based learning is a viable and portable alternative to manual rule engineering for coreference resolution. By utilizing domain-independent features and a standard decision tree algorithm, the authors established a foundation for the data-driven approaches that dominate the field today.

Summary updated

BibTeX
@inproceedings{soon1999corpus,
  author = {Wee Meng Soon and Hwee Tou Ng and Chung Yong Lim},
  title = {{Corpus-Based Learning for Noun Phrase Coreference Resolution}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {1999},
  url = {https://openreview.net/forum?id=QOXO7ufm1u8},
}
Figure