The Same-Head Heuristic for Coreference

Micha Elsner, Eugene CharniakACL 2010

Abstract

Coreference resolution often relies on the "same-head heuristic," assuming that noun phrases sharing the same head noun are coreferent. While effective on gold mention datasets, this heuristic fails in realistic scenarios involving automatic mention detection. We present an unsupervised generative model that uses syntactic features to distinguish between coreferent and non-coreferent same-head pairs. Our results on the MUC-6 corpus demonstrate significant improvements in precision and CEAF scores in realistic settings, highlighting the limitations of standard heuristics and evaluation metrics.

Introduction

In coreference resolution, the task is to identify mentions in a text that refer to the same real-world entity. A common rule of thumb is the "same-head heuristic": if two noun phrases (NPs) share the same head noun (e.g., "the company" and "the company"), they are likely coreferent. However, this assumption is often a byproduct of artificial experimental setups using "gold mentions"—human-curated lists of relevant NPs. In realistic systems that must detect mentions from raw text, the heuristic frequently fails. For instance, "recent employees" and "employees who have worked for longer" share a head but refer to distinct groups. This paper explores an unsupervised approach to learning when not to link same-head NPs.

Methodology

We propose an unsupervised generative model based on the IBM Model 2 framework, treating coreference as an alignment problem. For each NP in a document, the model selects either a previous NP as an antecedent or a "null" antecedent, indicating the introduction of a new entity.

To overcome the lack of labeled data, the model uses pronoun bootstrapping. Since pronouns (e.g., "it", "they") typically refer to nearby, salient NPs, training on these pairs allows the model to learn syntactic environments and distances characteristic of coreferent relationships. A feature-rich log-linear model then evaluates potential links based on:

  • Distance: Sentence and word-count gaps between mentions.
  • Syntax: Syntactic roles and phrase modifiers (e.g., "PP-of" modifiers).
  • Head Type: Classification as proper noun, common noun, or pronoun.

The final system is a mixture model that distinguishes between coreferent pairs and non-coreferent pairs that happen to share a head, effectively breaking the symmetry of the same-head heuristic.

Evaluation and Results

We evaluated the model on the MUC-6 coreference corpus across three settings: Gold Mentions, NPs (automatically detected base noun phrases), and Nouns (all detected nouns).

Table 1: Performance comparison of the Alignment model against the "Link all" baseline across different mention detection settings.

As shown in Figure 1, the "Alignment" model significantly outperforms the "Link all" baseline in realistic settings (NPs and Nouns). In the NP setting, the model's precision ($b^3$ pr) reached 87.2, compared to 67.2 for the baseline. Furthermore, the CEAF score—a metric more sensitive to spurious links than $b^3$—improved from 62.2 to 67.0.

Interestingly, the model performs worse than the baseline on gold mentions. This confirms that the same-head heuristic's perceived strength is a "mirage" caused by human annotators filtering out non-coreferent same-head pairs. In the real world, these pairs constitute significant noise that our model successfully filters.

Discussion

The results highlight a critical precision-recall tradeoff. By being more selective, the model improves precision but misses some links, leading to lower recall and $b^3$ F-scores. This sensitivity underscores the importance of metric choice; $b^3$ tends to reward "link-all" strategies, whereas CEAF provides a more balanced assessment of high-precision approaches. Manual analysis revealed that common exceptions to the same-head heuristic often involve specific modifiers, time/measure phrases, or partitives.

Conclusion

Moving beyond simple heuristics is essential for robust coreference resolution in raw text. Our unsupervised generative model demonstrates that syntactic features can effectively distinguish coreferent same-head pairs from coincidental ones. Future work should focus on balancing the precision-recall tradeoff and refining the model's handling of proper noun abbreviations.

Summary updated

BibTeX
@inproceedings{elsner2010same,
  author = {Micha Elsner and Eugene Charniak},
  title = {{The Same-Head Heuristic for Coreference}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
}
Figure