Abstract
Classical coreference systems encode various syntactic, discourse, and semantic phenomena explicitly, using heterogenous features computed from hand-crafted heuristics. In contrast, we present a state-of-the-art coreference system that captures such phenomena implicitly, with a small number of homogeneous feature templates examining shallow properties of mentions. Surprisingly, our features are actually more effective than the corresponding hand-engineered ones at modeling these key linguistic phenomena, allowing us to win “easy victories” without crafted heuristics. These features are successful on syntax and discourse; however, they do not model semantic compatibility well, nor do we see gains from experiments with shallow semantic features from the literature, suggesting that this approach to semantics is an “uphill battle.” Nonetheless, our final system outperforms the Stanford system (Lee et al. (2011), the winner of the CoNLL 2011 shared task) by 3.5% absolute on the CoNLL metric and outperforms the IMS system (Björkelund and Farkas (2012), the best publicly available English coreference system) by 1.9% absolute.
Introduction
Coreference resolution—the task of identifying all mentions in a text that refer to the same entity—is a foundational challenge in Natural Language Processing (NLP). Historically, researchers addressed this by building "heterogeneous" systems: complex pipelines filled with specific heuristics for different types of words (e.g., one set of rules for pronouns, another for definite descriptions). These systems are often difficult to tune and brittle. This paper proposes a "homogeneous" approach that learns linguistic patterns directly from data using simple, repetitive feature templates within a mention-ranking framework.
The Mention-Ranking Framework
The authors propose a mention-synchronous (or mention-ranking) framework. For every mention in a document, the model must make a single choice: link it to one of the preceding mentions or declare it "NEW" (the start of a new entity chain).
As shown in the figure above, this framework allows the model to explicitly handle various scenarios, such as correctly identifying a new entity ($a_1$) or navigating errors like "False New" ($a_2$), where an anaphoric mention is incorrectly labeled as new, and "False Anaphor" ($a_3$), where a new entity is incorrectly linked to a previous mention.
Feature Engineering and "Easy Victories"
The core of the approach is the use of surface features and conjunctions. Instead of writing complex rules for syntax or discourse, the model uses simple templates that look at shallow properties like the first word of a mention or its distance from a candidate antecedent.
By conjoining these simple features (e.g., [Mention Distance] + [Word Identity]), the model implicitly learns the linguistic patterns that previous researchers had to code by hand. For example, it can learn that "she" refers to female entities by observing its correlation with other female-gendered mentions in the training data, achieving "easy victories" in syntax and discourse without explicit heuristics.
The "Uphill Battle" of Semantics
While the model excelled at syntax, it struggled with semantics. Surface features cannot easily resolve mentions with different heads that require world knowledge (e.g., linking "the executive" to "the CEO"). Adding external knowledge from WordNet or gender/number databases provided only marginal gains. The authors identify a statistical bias: in real-world data, there are far more non-referential mentions than real ones. This creates a high bar for semantic signals; a weak synonymy link is often not strong enough to overcome the model's hesitation to create a link, as the risk of a "false positive" is too high.
Key Results
The Berkeley system set a new state-of-the-art record on the CoNLL 2011 shared task benchmark:
- Performance: It outperformed the Stanford system (the CoNLL 2011 winner) by 3.5% absolute on the CoNLL metric.
- Comparison: It beat the IMS system (the best publicly available system at the time) by 1.9% absolute.
- Efficiency: By resolving mentions in a single pass without complex multi-pass sieves, the system is significantly faster and easier to train than its predecessors.
Conclusion
The success of this approach suggests that many aspects of coreference resolution are more effectively modeled through data-driven surface features than hand-crafted rules. However, the "uphill battle" of semantics remains. Future work must find ways to integrate deeper semantic compatibility and world knowledge to move beyond the limitations of shallow NLP.
Summary updated