An NP-Cluster Based Approach to Coreference Resolution

Xiaofeng Yang, Jian Su, GuoDong Zhou, Chew Lim TanCOLING 2004

TL;DR

This paper introduces an NP-Cluster based approach to coreference resolution, moving beyond pairwise NP comparisons to leverage the richer information available in coreferential clusters. By framing coreference as linking an NP to a cluster rather than another NP, the authors achieve significant improvements in both recall and precision on the MEDLINE dataset.

What Problem Does This Paper Address?

Coreference resolution, the task of identifying expressions that refer to the same entity in a text, is a crucial step in natural language understanding. Traditional supervised machine learning approaches often frame this as a binary classification problem: determining if two noun phrases (NPs) are coreferential. However, a major limitation of this NP-NP approach is that individual NPs often lack sufficient descriptive information to reliably determine their referents. This can lead to difficulties in learning accurate rules and making correct resolutions, especially when dealing with pronouns or NPs with limited context. The paper aims to overcome this by utilizing the collective information within coreferential clusters.

Methods

The authors propose an "NP-Cluster based approach" that reframes coreference resolution. Instead of learning to link an NP to another NP, the system learns to link an NP to an existing coreferential cluster.

Key aspects of the method:

  • Instance Representation: An instance is formed by an NP, a coreferential cluster, and a reference NP from that cluster. The label is positive if the NP belongs to the cluster, negative otherwise.
  • Feature Engineering: The approach extends the baseline NP-NP features with six new features that capture the relationship between an NP and a cluster. These include:
  • Cluster_NumAgree, Cluster_GenAgree, Cluster_SemAgree: Agreement in number, gender, and semantic class between the NP and cluster elements.
  • Cluster_Length: The number of NPs in the cluster, reflecting the entity's salience.
  • Cluster_StrSim: String similarity between the NP and the union of tokens in the cluster.
  • Cluster_StrLNPSim: String similarity between the NP and the longest NP in the cluster, hypothesizing that longer NPs provide richer descriptions.
  • Training and Resolution: A C5.0 decision tree classifier is trained on these NP-cluster instances. During resolution, an NP is paired with existing clusters, and the classifier predicts the likelihood of the NP belonging to each cluster. Two strategies, Most Recent First (MRF) and Best First (BF), are used to select the best cluster for an NP.

Datasets

  • MEDLINE: The experiments were conducted on a subset of the MEDLINE dataset, specifically 228 abstracts from the GENIA dataset.
  • 70 documents were used for training.
  • 30 documents were used for testing.
  • The average document length is 244 words.
  • A characteristic noted is the low ratio of pronouns (around 3%) compared to newswire domains.

Key Insights and Ideas

  • Clusters Offer Richer Information: The core idea is that coreferential clusters, by aggregating information from multiple mentions, provide a more comprehensive description of an entity than any single NP. This richer context is crucial for accurate coreference resolution.
  • Beyond Pairwise Comparisons: The NP-Cluster approach moves beyond the limitations of pairwise NP comparisons, which can be hindered by the inherent lack of descriptive power in individual NPs.
  • Feature Importance: String similarity features (both NP-NP and NP-Cluster) are identified as highly important for coreference resolution, supporting previous findings. The introduction of cluster-specific string similarity features (Cluster_StrSim, Cluster_StrLNPSim) significantly boosts performance.
  • Salience Matters: Cluster_Length is introduced as a feature to capture entity salience, though its impact on performance was less pronounced than string similarity features.
  • BF Strategy Superiority: The Best First (BF) antecedent selection strategy consistently outperformed the Most Recent First (MRF) strategy across all tested systems.

Key Results

The NP-Cluster based approach demonstrated significant improvements over the baseline NP-NP approach on the MEDLINE dataset:

  • Recall: Improved by 4.6%.
  • Precision: Improved by 1.3%.
  • F-measure: Increased by 2.8% (from 78.9% for the baseline to 81.7% for the NP-Cluster approach using BF strategy).

The paper also compared their approach to an "AllAnte" system, which attempted to incorporate cluster information by pairing an NP with all its antecedents. This yielded a recall boost but a significant precision drop, highlighting the effectiveness of the NP-Cluster framework.

Limitations and Future Work

  • Limited Context: The current approach resolves an NP using only clusters identified before it in the text. Information from NPs appearing after the current NP could potentially improve resolution, but is not utilized.
  • Cluster Accuracy: The system relies on pre-annotated coreferential clusters during training. In real-world scenarios, these clusters might not be perfectly accurate, which could affect the learned model.
  • Future Work: The authors plan to develop more robust clustering algorithms and explore ways to link an NP to a "globally best cluster," potentially by considering future information or improving the clustering process itself.

Summary updated

BibTeX
@inproceedings{yang2004np,
  author = {Xiaofeng Yang and Jian Su and GuoDong Zhou and Chew Lim Tan},
  title = {{An NP-Cluster Based Approach to Coreference Resolution}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2004},
}
Figure