Introduction
Pronoun anaphora resolution—the task of linking pronouns like "it" or "him" to previously mentioned entities—is a fundamental challenge in natural language processing. Traditional approaches often rely on pairwise models that evaluate mentions in isolation. However, these models frequently suffer from logical inconsistencies, such as predicting that entity A matches B and B matches C, yet A does not match C.
While ranking and graph-partitioning models have been proposed to address these issues, they often lack the incremental, sequential nature of human language processing. This paper presents a supervised system based on Factorial Hidden Markov Models (FHMMs) that mimics human short-term memory. By maintaining a dynamic "antecedent buffer," the system treats resolution as a continuous discourse process, significantly improving accuracy and efficiency.
Methodology: The Factorial Hidden Markov Model
The core innovation of this system is the application of an FHMM to track the state of a discourse. Unlike standard HMMs with a single hidden state, an FHMM utilizes multiple independent hidden variables that interact to generate observed words.
The Antecedent Buffer
To manage computational complexity and reflect linguistic theory, the model maintains a "short-term memory" buffer of the six most recent entities. Corpus analysis reveals that this limited window accounts for 99.2% of all pronoun references. This approach avoids the need to compare a pronoun against every preceding mention in a document.
Model Architecture
At each time step (word), the model tracks three primary hidden variables:
- Operation ($op$): Determines if a word introduces a new entity, refers to an old one in the buffer, or is none (not a mention).
- Coreference Features ($cr$): Tracks attributes such as gender (male, female, neutral), number (singular, plural), and entity type (person, organization, etc.).
- Part-of-Speech ($pos$): Captures the grammatical category to leverage linguistic constraints like binding principles.
The model calculates transition probabilities between these states. When an old entity is selected, the buffer is reordered to move that entity to the top, modeling "salience." The observation model then uses these features to predict the most likely word. The authors found that treating features like gender and personhood as independent events rather than complex joint distributions significantly reduced data sparsity and improved overall model performance.
Experimental Evaluation
The system was evaluated using the ACE (Automatic Context Extraction) Phase 2 corpus, which includes three genres: Newspaper articles (NPAPER), Newswire (NWIRE), and Broadcast News (BNEWS). Performance was compared against several baselines, including the unsupervised emPronouns system and the state-of-the-art RK (Ranking) model.
Key Results
The FHMM system consistently outperformed all baseline models across every genre in the ACE dataset.
As shown in the results, the system achieved its highest performance on the NPAPER dataset (79.4%). This is likely due to the longer, more explicit coreference chains found in newspaper articles, which benefit from the FHMM’s ability to track discourse state sequentially over time.
Discussion and Future Work
While the FHMM approach is highly effective, several challenges remain. The fixed buffer size, while efficient, cannot resolve the 0.8% of pronouns referring to very distant antecedents. Additionally, the model struggles with plural ambiguity in English (where gender distinctions are lost) and complex coordinate noun phrases (e.g., "A and B").
Future research will focus on extending this framework to general coreference resolution beyond pronouns and integrating deeper syntactic parsing directly into the FHMM transition model to better capture linguistic constraints.
Summary updated