TL;DR
This paper introduces a "twin-model" for coreference resolution, which simultaneously models the likelihood of a mention being coreferential with an existing entity and the probability of it being a new, unlinked entity. This approach outperforms traditional thresholding methods and eliminates the need for manual parameter tuning, offering a more robust and portable solution.
What Problem Does This Paper Address?
Coreference resolution is the task of identifying all mentions (words or phrases) in a text that refer to the same real-world entity. For example, "John," "he," and "his" might all refer to the same person. Existing machine learning approaches typically focus on determining if a new mention (an "anaphor") is coreferential with a candidate antecedent (a previously seen mention that could be the same entity). However, a crucial and often overlooked problem is when a mention is not coreferential with any existing candidate. This can happen if the mention is truly new to the discourse or if it's the first mention in a coreference chain. Previous methods often handled this by simply setting a score threshold, which is suboptimal and requires manual tuning. Other approaches tried to use separate anaphoricity classifiers, but these were often trained independently.
Methods
The paper proposes a "twin-model" consisting of two components, both implemented using log-linear (maximum entropy) models:
- Link Model (Pi(L|ej, mt)): This component is standard in coreference resolution. It models the probability that a current mention ($m_t$) is coreferential with a specific existing entity ($e_j$). This involves features characterizing the relationship between the mention and the entity.
- Creation Model (Pc(C|Et, mt)): This is the novel contribution. It models the probability that the current mention ($m_t$) is not coreferential with any of the existing entities ($E_t$) in the discourse. This component explicitly handles the "coreference or not" decision.
A key challenge for the creation model is its dependency on all preceding entities ($E_t$), which can be computationally expensive. To address this, the authors impose constraints on feature functions. They propose that features in the creation model can be efficiently computed from corresponding features in the link model. Specifically, features in the link model are separable into functions of the mention-entity pair and the prediction (coreferential or not), while creation model features depend on the values of link model features across all entities in $E_t$, rather than the entities themselves. This allows for efficient computation.
Datasets
The proposed twin-model was evaluated on the ACE 2005 dataset. This dataset comprises documents from various sources like newswires, web logs, and telephone conversations. The authors split the data into training and testing sets, reserving the last 16% of documents from each source for testing.
Key Insights and Ideas
- Explicitly modeling "not coreferential" is crucial: The paper argues that treating the "not coreferential" case as a separate, explicitly modeled event, rather than a mere score threshold, leads to better performance.
- Twin-model synergy: The link and creation models are designed to work together and are trained simultaneously. This tight coupling allows the creation model to leverage information from the link model features, making it more efficient and effective.
- Eliminating manual threshold tuning: The creation model inherently handles the decision of whether to link or create a new entity, removing the need for a free parameter that requires manual tuning for different tasks or datasets. This makes the model more portable.
- Feature engineering for efficiency: The constraints on feature functions for the creation model are key to making the approach computationally feasible, allowing it to be trained and used effectively.
Key Results
The twin-model consistently outperformed a thresholding baseline on the ACE 2005 dataset, both in terms of entity F-measure and ACE-Value. For instance, at a threshold of 0.9 for the baseline, the twin-model achieved an entity F-measure of 0.848 compared to the baseline's 0.591 (a 43.5% difference). The ACE-Value also showed improvement, though less dramatically. The paper highlights that the optimal threshold for the baseline varied depending on the evaluation metric (entity F-measure vs. ACE-Value), demonstrating the inflexibility of the thresholding approach. The twin-model, by contrast, achieved its best performance without any threshold tuning.
Limitations and Future Work
The paper acknowledges that the creation model's conditioning on all preceding entities can still be computationally intensive, although their proposed constraints mitigate this. The authors suggest that further research could explore more sophisticated ways to model the relationship between the creation model and the link model features. The paper also implicitly suggests that applying this twin-model to other coreference resolution tasks or datasets would be a natural next step.
Summary updated