Introduction
Coreference resolution—the task of determining when different words or phrases in a text refer to the same entity—remains a fundamental challenge in Natural Language Processing (NLP). Identifying that "Barack Obama," "the former president," and "him" all point to the same individual requires significant linguistic and world knowledge. At the CoNLL-2012 Shared Task, a team from the Harbin Institute of Technology proposed a "Mixed Deterministic Model" designed to bridge the gap between rule-based intuition and machine learning efficiency across English and Chinese.
Methodology
The proposed model utilizes a two-stage pipeline consisting of mention detection followed by coreference resolution. The system is characterized as "mixed" because it integrates deterministic rules with targeted machine learning components.
The Multi-Pass Sieve
The core of the resolution engine is a multi-pass sieve, inspired by the Stanford sieve system. It applies a series of rules in order of decreasing precision:
- Exact Match: Mentions are linked if they match exactly after removing stop words like "the."
- Partial Match: Mentions sharing a headword (e.g., "the leader of people" and "leader") are linked.
- Pronoun Resolution: A hierarchy of person (1st, 2nd, 3rd), gender, and number is used to identify the most likely antecedents for pronouns.
Targeted Machine Learning for "It"
While most of the system is rule-based, the authors employed a C4.5 Decision Tree specifically to handle the English pronoun "it." This pronoun is notoriously difficult because it can be referential (pointing to a noun) or pleonastic (used in impersonal constructions like "It is 5 PM"). The classifier uses features such as Part-of-Speech tags and local context to distinguish between these uses.
Multilingual Adaptation
To handle Chinese, which lacks explicit gender and number markers, the authors constructed custom PERSON dictionaries and gender/number classifiers. These were developed by mining the training data for honorifics and appellations (e.g., "Mr.", "Professor") to infer entity types and gender.
Experimental Results
The system was evaluated on the OntoNotes corpus for the CoNLL-2012 Shared Task. Performance was measured using the average F1 score across three standard metrics: MUC, B³, and CEAF.
Overall Performance
The model achieved competitive results in the "closed" track, where external resources were prohibited. For the English Closed task, the system reached an average F1 score of 58.68.
In the Chinese tasks, the model achieved an F1 of 60.69 in the Closed track and 61.02 in the Open track. Interestingly, the inclusion of external tools in the Open track provided only marginal gains, likely due to noise introduced by segmentation mismatches between external parsers and the official dataset.
Analysis of the "It" Classifier
The specialized classifier for the pronoun "it" proved effective in reducing noise within coreference chains.
As shown in Table 1, the classifier achieved a total F-score of 0.68. While not perfect, this targeted machine learning approach significantly improved the handling of one of the most ambiguous pronouns in the English language.
Discussion: The Mention Detection Bottleneck
A key insight from the evaluation is that mention detection serves as a significant bottleneck. When the system was tested using "Gold" (perfect) mention boundaries, the Chinese F1 score increased dramatically to 76.05. This suggests that while the coreference linking rules are robust, the initial identification of candidate mentions remains a primary source of error.
Conclusion
The mixed deterministic approach demonstrates that combining high-precision rules with targeted machine learning can effectively address complex NLP tasks like coreference resolution. Future work will focus on incorporating deeper semantic features, such as predicate-argument structures, and improving the alignment of external tools to reduce noise in multilingual contexts.
Reference: Yuan, B., Chen, Q., et al. (2012). A Mixed Deterministic Model for Coreference Resolution. Proceedings of the Joint Conference on EMNLP and CoNLL: Shared Task.
Summary updated