Simple Coreference Resolution with Rich Syntactic and Semantic Features

Aria Haghighi, Dan KleinEMNLP 2009 · OpenReview

Abstract

Coreference systems are driven by syntactic, semantic, and discourse constraints. We present a simple approach which completely modularizes these three aspects. In contrast to much current work, which focuses on learning and on the discourse component, our system is deterministic and is driven entirely by syntactic and semantic compatibility as learned from a large, unlabeled corpus. Despite its simplicity and discourse naivete, our system substantially outperforms all unsupervised systems and most supervised ones. Primary contributions include (1) the presentation of a simple-to-reproduce, high-performing baseline and (2) the demonstration that most remaining errors can be attributed to syntactic and semantic factors external to the coreference phenomenon (and perhaps best addressed by non-coreference systems).

Introduction

Coreference resolution—the task of determining which mentions in a text (like "Barack Obama," "the president," and "he") refer to the same real-world entity—is a fundamental challenge in natural language processing. Traditionally, this problem has been approached by building complex, joint models that attempt to learn how to weigh dozens of "weak" features using supervised data.

Researchers from UC Berkeley demonstrate that a simple, deterministic coreference resolution system—one that relies on high-quality syntactic and semantic features rather than complex machine learning—can outperform all unsupervised systems and rival the state-of-the-art supervised models of its time. The paper suggests that the bottleneck in coreference isn't how we model discourse, but rather how well we understand the underlying syntax and semantics of the text.

Methodology

The authors propose a three-step deterministic pipeline that avoids the need for supervised coreference data by focusing on modularity and feature richness.

Syntactic Module and Tree Distance

Instead of measuring the "flat" distance between words in a sentence, the system uses an augmented parser to calculate tree distance. This allows the model to apply strict linguistic constraints like the "i-within-i" filter (which prevents a noun phrase from referring to one of its own sub-components) and identifies specific structures like appositives and predicate nominatives.

Syntactic tree showing coreference between 'Nintendo of America' and 'its' via tree distance.

Semantic Module

To handle cases where mentions have different heads (like "AOL" and "company"), the authors automatically mine "compatibility lists" from massive unlabeled datasets, including Wikipedia and newswire. They use a bootstrapping technique to find syntactic patterns that imply coreference, such as "[X] who served as [Y]."

Handling Role Appositives

A notable innovation is the handling of "role appositives" (e.g., "[painter] Pablo Picasso"). Standard parsers often produce flat structures for these, so the authors heuristically inject internal NP structure to correctly identify these high-confidence coreference links.

Comparison of (a) standard flat parser output and (b) the authors' augmented structure for role appositives.

Selection

After filtering out impossible candidates based on syntax and semantics, the system simply picks the "closest" remaining antecedent based on tree distance.

Key Results

The system, despite being deterministic and largely unsupervised for the coreference task itself, achieved impressive results on standard benchmarks:

  • ACE2004: On the Culotta test split, it achieved a 79.0 $b^3$ F1 score, outperforming the best unsupervised systems of the time (70.9) and coming within 2 points of the top supervised systems (80.8).
  • MUC-6: Outperformed the state-of-the-art unsupervised system by over 2 points in MUC F1.
  • Ablation Study: Adding semantic compatibility alone improved the pairwise F1 on ACE2004 from 48.3 to 57.5, proving that "rich" features are the primary driver of performance.

Discussion and Insights

  • Syntax is the Bottleneck: Most failures were caused by simple parsing mistakes or missing semantic knowledge (e.g., not knowing that "pay" and "wage" are related), rather than a lack of complex discourse modeling.
  • Tree Distance > Flat Distance: Measuring distance through the branches of a parse tree is a much more effective proxy for "salience" than simply counting the number of words between mentions.
  • The Power of Unlabeled Data: Knowledge about entity types (e.g., "Microsoft" is a "company") can be effectively extracted from the way people write in the wild, reducing the need for expensive hand-annotated data.

Conclusion

By treating syntax and semantics as independent filters, the system remains interpretable and easy to debug compared to "black-box" supervised learners. The authors suggest that instead of building more complex coreference models, the community should focus on building better parsers and more comprehensive semantic knowledge bases.

Summary updated

Abstract

Coreference systems are driven by syntactic, semantic, and discourse constraints. We present a simple approach which completely modularizes these three aspects. In contrast to much current work, which focuses on learning and on the discourse component, our system is deterministic and is driven entirely by syntactic and semantic compatibility as learned from a large, unlabeled corpus. Despite its simplicity and discourse naivete, our system substantially outperforms all unsupervised systems and most supervised ones. Primary contributions include (1) the presentation of a simple-to-reproduce, high-performing baseline and (2) the demonstration that most remaining errors can be attributed to syntactic and semantic factors external to the coreference phenomenon (and perhaps best addressed by non-coreference systems).
BibTeX
@inproceedings{haghighi2009simple,
  author = {Aria Haghighi and Dan Klein},
  title = {{Simple Coreference Resolution with Rich Syntactic and Semantic Features}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2009},
  url = {https://openreview.net/forum?id=BJbCn-MuZS},
}
Figure