Abstract
We propose a new deterministic approach to coreference resolution that combines the global information and precise features of modern machine-learning models with the transparency and modularity of deterministic, rule-based systems. Our sieve architecture applies a battery of deterministic coreference models one at a time from highest to lowest precision, where each model builds on the previous model's cluster output. The two stages of our sieve-based architecture, a mention detection stage that heavily favors recall, followed by coreference sieves that are precision-oriented, offer a powerful way to achieve both high precision and high recall. Further, our approach makes use of global information through an entity-centric model that encourages the sharing of features across all mentions that point to the same real-world entity. Despite its simplicity, our approach gives state-of-the-art performance on several corpora and genres, and has also been incorporated into hybrid state-of-the-art coreference systems for Chinese and Arabic. Our system thus offers a new paradigm for combining knowledge in rule-based systems that has implications throughout computational linguistics.
Introduction
Coreference resolution—the task of linking mentions in a text that refer to the same real-world entity—is a cornerstone of natural language understanding. Traditionally, this task has been approached through complex machine learning (ML) models. However, these models often act as "black boxes," requiring vast amounts of labeled data and proving difficult to tune or generalize across domains. This paper presents a deterministic, rule-based alternative that utilizes a "sieve" architecture to achieve state-of-the-art performance with greater transparency and modularity.
Methodology
The proposed system operates in two primary stages: high-recall mention detection followed by a sequence of precision-ranked coreference sieves.
Mention Detection
The first stage identifies all potential mentions, including noun phrases (NPs), pronouns, and named entities. To ensure high recall, the system initially identifies a broad set of candidates and then applies filters to remove non-referential items, such as pleonastic "it" (e.g., "It is raining") or partitives.
The Sieve Architecture
The core of the system is a multi-pass architecture consisting of ten independent sieves. These sieves are applied in order from highest to lowest precision. This "easy-first" strategy allows the system to make certain decisions early, building high-quality entity clusters that inform more difficult decisions in later passes.
Key sieves include:
- Speaker Identification: Matches speakers to first-person pronouns in quotes.
- String Match: Exact and relaxed string matching.
- Precise Constructs: Identifies syntactic structures like appositives and predicate nominatives.
- Head Match: Various passes matching the "head" word of phrases while ensuring compatibility of modifiers.
- Pronoun Resolution: The final pass for resolving remaining pronouns.
Entity-Centric Modeling
Unlike traditional models that compare pairs of mentions, this system compares a mention to an entire cluster of previously linked mentions. This "entity-centric" approach allows the system to aggregate attributes (gender, number, animacy) across all mentions in a cluster. For example, if a cluster contains "Mr. Smith," the system "knows" the entity is male, preventing it from incorrectly linking "she" to that cluster in a later sieve.
Evaluation and Results
The system was evaluated on major benchmarks, including OntoNotes (CoNLL-2011), ACE 2004, and MUC-6. It achieved state-of-the-art results, winning the CoNLL-2011 shared task.
Ablation studies highlighted the necessity of the architecture's design:
- Multi-pass impact: Running all rules simultaneously instead of in a ranked sequence resulted in a 6-point drop in F1 score.
- Entity-centric impact: Disabling feature sharing across clusters led to a 3.4-point decrease in performance.
Furthermore, the system's modularity allowed for successful adaptation to Chinese and Arabic, demonstrating that deterministic linguistic rules can be robust across different languages.
Conclusion
This work demonstrates that a well-structured, deterministic approach can outperform complex statistical models in coreference resolution. By prioritizing precision and leveraging global entity information through a multi-pass sieve architecture, the system provides a powerful, transparent, and modular framework for NLP. Future work remains in addressing deep semantic challenges where world knowledge is required to resolve ambiguous references.
Summary updated